1. 20 Aug, 2015 1 commit
  2. 11 Aug, 2015 1 commit
  3. 04 Aug, 2015 1 commit
    • Richard Scothern's avatar
      Add pull through cache functionality to the Registry which can be configured · 94935f39
      Richard Scothern authored
      with a new `proxy` section in the configuration file.
      
      Create a new registry type which delegates storage to a proxyBlobStore
      and proxyManifestStore.  These stores will pull through data if not present
      locally.  proxyBlobStore takes care not to write duplicate data to disk.
      
      Add a scheduler to cleanup expired content. The scheduler runs as a background
      goroutine.  When a blob or manifest is pulled through from the remote registry,
      an entry is added to the scheduler with a TTL.  When the TTL expires the
      scheduler calls a pre-specified function to remove the fetched resource.
      
      Add token authentication to the registry middleware.  Get a token at startup
      and preload the credential store with the username and password supplied in the
      config file.
      
      Allow resumable digest functionality to be disabled at runtime and disable
      it when the registry is a pull through cache.
      Signed-off-by: default avatarRichard Scothern <richard.scothern@gmail.com>
      94935f39
  4. 30 Jul, 2015 2 commits
  5. 29 Jul, 2015 1 commit
  6. 24 Jul, 2015 1 commit
    • Richard's avatar
      Manifest and layer soft deletion. · 9c1dd694
      Richard authored
      
      Implement the delete API by implementing soft delete for layers
      and blobs by removing link files and updating the blob descriptor
      cache.  Deletion is configurable - if it is disabled API calls
      will return an unsupported error.
      
      We invalidate the blob descriptor cache by changing the linkedBlobStore's
      blobStatter to a blobDescriptorService and naming it blobAccessController.
      
      Delete() is added throughout the relevant API to support this functionality.
      Signed-off-by: default avatarRichard Scothern <richard.scothern@gmail.com>
      9c1dd694
  7. 21 Jul, 2015 1 commit
  8. 11 Jun, 2015 3 commits
  9. 31 May, 2015 1 commit
    • xiekeyang's avatar
      Feature: Web Panic Reporting via hooks · 47aa47e3
      xiekeyang authored
      
      This PR is for issue of "email after registry webapp panic" #41, improving my
      previous design (closed).
      It use self setting up hooks, to catch panic in web application.
      And, send email in hooks handle directly, to no use new http server and
      handler.
      Signed-off-by: default avatarxiekeyang <keyangxie@126.com>
      47aa47e3
  10. 16 May, 2015 1 commit
    • Stephen J Day's avatar
      Refactor Blob Service API · 593bbccd
      Stephen J Day authored
      
      This PR refactors the blob service API to be oriented around blob descriptors.
      Identified by digests, blobs become an abstract entity that can be read and
      written using a descriptor as a handle. This allows blobs to take many forms,
      such as a ReadSeekCloser or a simple byte buffer, allowing blob oriented
      operations to better integrate with blob agnostic APIs (such as the `io`
      package). The error definitions are now better organized to reflect conditions
      that can only be seen when interacting with the blob API.
      
      The main benefit of this is to separate the much smaller metadata from large
      file storage. Many benefits also follow from this. Reading and writing has
      been separated into discrete services. Backend implementation is also
      simplified, by reducing the amount of metadata that needs to be picked up to
      simply serve a read. This also improves cacheability.
      
      "Opening" a blob simply consists of an access check (Stat) and a path
      calculation. Caching is greatly simplified and we've made the mapping of
      provisional to canonical hashes a first-class concept. BlobDescriptorService
      and BlobProvider can be combined in different ways to achieve varying effects.
      
      Recommend Review Approach
      -------------------------
      
      This is a very large patch. While apologies are in order, we are getting a
      considerable amount of refactoring. Most changes follow from the changes to
      the root package (distribution), so start there. From there, the main changes
      are in storage. Looking at (*repository).Blobs will help to understand the how
      the linkedBlobStore is wired. One can explore the internals within and also
      branch out into understanding the changes to the caching layer. Following the
      descriptions below will also help to guide you.
      
      To reduce the chances for regressions, it was critical that major changes to
      unit tests were avoided. Where possible, they are left untouched and where
      not, the spirit is hopefully captured. Pay particular attention to where
      behavior may have changed.
      
      Storage
      -------
      
      The primary changes to the `storage` package, other than the interface
      updates, were to merge the layerstore and blobstore. Blob access is now
      layered even further. The first layer, blobStore, exposes a global
      `BlobStatter` and `BlobProvider`. Operations here provide a fast path for most
      read operations that don't take access control into account. The
      `linkedBlobStore` layers on top of the `blobStore`, providing repository-
      scoped blob link management in the backend. The `linkedBlobStore` implements
      the full `BlobStore` suite, providing access-controlled, repository-local blob
      writers. The abstraction between the two is slightly broken in that
      `linkedBlobStore` is the only channel under which one can write into the global
      blob store. The `linkedBlobStore` also provides flexibility in that it can act
      over different link sets depending on configuration. This allows us to use the
      same code for signature links, manifest links and blob links.  Eventually, we
      will fully consolidate this storage.
      
      The improved cache flow comes from the `linkedBlobStatter` component
      of `linkedBlobStore`. Using a `cachedBlobStatter`, these combine together to
      provide a simple cache hierarchy that should streamline access checks on read
      and write operations, or at least provide a single path to optimize. The
      metrics have been changed in a slightly incompatible way since the former
      operations, Fetch and Exists, are no longer relevant.
      
      The fileWriter and fileReader have been slightly modified to support the rest
      of the changes. The most interesting is the removal of the `Stat` call from
      `newFileReader`. This was the source of unnecessary round trips that were only
      present to look up the size of the resulting reader. Now, one must simply pass
      in the size, requiring the caller to decide whether or not the `Stat` call is
      appropriate. In several cases, it turned out the caller already had the size
      already. The `WriterAt` implementation has been removed from `fileWriter`,
      since it is no longer required for `BlobWriter`, reducing the number of paths
      which writes may take.
      
      Cache
      -----
      
      Unfortunately, the `cache` package required a near full rewrite. It was pretty
      mechanical in that the cache is oriented around the `BlobDescriptorService`
      slightly modified to include the ability to set the values for individual
      digests. While the implementation is oriented towards caching, it can act as a
      primary store. Provisions are in place to have repository local metadata, in
      addition to global metadata. Fallback is implemented as a part of the storage
      package to maintain this flexibility.
      
      One unfortunate side-effect is that caching is now repository-scoped, rather
      than global. This should have little effect on performance but may increase
      memory usage.
      
      Handlers
      --------
      
      The `handlers` package has been updated to leverage the new API. For the most
      part, the changes are superficial or mechanical based on the API changes. This
      did expose a bug in the handling of provisional vs canonical digests that was
      fixed in the unit tests.
      
      Configuration
      -------------
      
      One user-facing change has been made to the configuration and is updated in
      the associated documentation. The `layerinfo` cache parameter has been
      deprecated by the `blobdescriptor` cache parameter. Both are equivalent and
      configuration files should be backward compatible.
      
      Notifications
      -------------
      
      Changes the `notification` package are simply to support the interface
      changes.
      
      Context
      -------
      
      A small change has been made to the tracing log-level. Traces have been moved
      from "info" to "debug" level to reduce output when not needed.
      Signed-off-by: default avatarStephen J Day <stephen.day@docker.com>
      593bbccd
  11. 27 Apr, 2015 1 commit
  12. 03 Apr, 2015 1 commit
  13. 01 Apr, 2015 1 commit
    • Stephen J Day's avatar
      Add redis pool to registry webapp · 3cad3c7b
      Stephen J Day authored
      
      Redis has been integrated with the web application for use with various
      services. The configuraiton exposes connection details, timeouts and pool
      parameters. Documentation has been updated accordingly.
      
      A few convenience methods have been added to the context package to get loggers
      with certain fields, exposing some missing functionality from logrus.
      Signed-off-by: default avatarStephen J Day <stephen.day@docker.com>
      3cad3c7b
  14. 25 Mar, 2015 1 commit
  15. 24 Mar, 2015 1 commit
  16. 03 Feb, 2015 2 commits
    • Stephen J Day's avatar
      Webhook notification support in registry webapp · 0a29b59e
      Stephen J Day authored
      
      Endpoints are now created at applications startup time, using notification
      configuration. The instances are then added to a Broadcaster instance, which
      becomes the main event sink for the application. At request time, an event
      bridge is configured to listen to repository method calls. The actor and source
      of the eventBridge are created from the requeest context and application,
      respectively. The result is notifications are dispatched with calls to the
      context's Repository instance and are queued to each endpoint via the
      broadcaster.
      
      This commit also adds the concept of a RequestID and App.InstanceID. The
      request id uniquely identifies each request and the InstanceID uniquely
      identifies a run of the registry. These identifiers can be used in the future
      to correlate log messages with generated events to support rich debugging.
      
      The fields of the app were slightly reorganized for clarity and a few horrid
      util functions have been removed.
      Signed-off-by: default avatarStephen J Day <stephen.day@docker.com>
      0a29b59e
    • Stephen J Day's avatar
      Add debug server to support pprof and expvar · 499382dd
      Stephen J Day authored
      
      If configured, a debug http server will be started to serve default registered
      endpoints, such as pprof and expvar. The endpoint should be secured carefully
      and not available to external traffic. It is disabled by default but the
      development config has been modified to make it available on localhost.
      Signed-off-by: default avatarStephen J Day <stephen.day@docker.com>
      499382dd
  17. 15 Dec, 2014 1 commit
  18. 11 Dec, 2014 1 commit
  19. 02 Dec, 2014 1 commit