Diego v0.1438.0

Changes from v0.1437.0 to v0.1438.0

• Depends on garden-linux-release v0.316.0.
• Depends on etcd-release 18.

Significant changes

• As a CF app developer, I would like to be able to stage Docker images from registries that present only the v2 registry API
• The BBS should retry a DesiredLRP update if it fails because of a CAS error
• Auction state requests should be resilient to network errors
• auctioneer should log the ids of the work it assigns to cell reps and the ids of any work rejected
• As a Diego operator, I would like to see the rep emit container creation time as a metric
• BBS should emit metrics for unclaimed, claimed, missing, and extra LRPs
• As a Diego developer, I would like a CONTRIBUTING document in diego-release instead of instructions in the main README
• As a Diego operator or developer, I would like instructions for deploying CF+Diego at various states of stability
• As a Diego team member, I would like to run the bbs benchmark suite in CI
• As a Diego developer, I would like a benchmark suite for retrieval of DesiredLRP and ActualLRP data by the route-emitter
• TPS listener should have logging around calls to external services
• Tune taskCBWorkPool and createActualMaxWorkers workpool sizes
• expose workpool sizes as command line arguments
• Investigate the time it takes to schedule task auctions for 4000 tasks
• cloudfoundry-incubator/diego-release #82: Allow skipping of acceptance tests by using regex
• cloudfoundry-incubator/diego-release #84: Re-enable btrfs and disk quotas on BOSH lite

BOSH job changes

None.

BOSH property changes

• Add acceptance_tests.skip_regex: Skip tests with descriptions matching this regular expression.
• Add benchmark-bbs.log_file: File name for benchmark log output.

Diego v0.1437.0

Changes from v0.1436.0 to v0.1437.0

• Depends on garden-linux-release v0.308.0.
• Depends on etcd-release 16.

Other significant changes

• As a Diego developer, I would like a benchmark suite for the nsync-bulker's retrieval of DesiredLRP data (in flight)
• As a Diego team member, I would like to run the bbs benchmark suite in CI (in flight)
• Include bosh-lite versions of manifests for the DATs and Diego Smoke Test errands in diego-release
• Bump cflinuxfs2 stack to 1.11.0+
• Auction state requests should be resilient to network errors
• Flakey diego-ssh/scp/scp_test
• cloudfoundry-incubator/rep #4: fixes the tests on windows

BOSH job changes

None.

BOSH property changes

• Add benchmark-bbs.log_level: Control log level of test suite in BBS benchmark errand.

Diego v0.1436.0

Changes from v0.1435.0 to v0.1436.0

• Depends on garden-linux-release v0.308.0.
• Depends on etcd-release 15.

Significant changes

• As a Diego operator, I would like the README to use customary methods of generating CF manifests for bosh-lite
• As a Diego developer, I would like a benchmark suite for the nsync-bulker's retrieval of DesiredLRP data
• BBS migration 1441411196 panics if deserializing encrypted data
• flake: encryption format.Encoding on unencrypted data
• Tune nf_conntrack_max value on Diego cells

BOSH job changes

• Add benchmark-bbs job for certain Diego benchmarks against the BBS API. As with the acceptance-tests and smoke-tests jobs, this is an errand that runs a test suite. It is not presently suitable to run against a production deployment, as the benchmark suite will alter the BBS database.

BOSH property changes

• Add properties under benchmark-bbs.* for the BBS-benchmarking errand.

Diego v0.1435.0

Changes from v0.1434.0 to v0.1435.0

• Depends on garden-linux-release v0.307.0.
• Depends on etcd-release 14.

Significant changes

• Pair with MEGA to incorporate etcd-release into diego-release
• As a Diego operator, I would like to allow developers to stage images from insecure private registries specified in my deployment manifest (in flight)
• As a Diego operator, I would like the README to use customary methods of generating CF manifests for bosh-lite
• Diego manifest generation should extract the director_uuid from the CF manifest
• Include bosh-lite versions of manifests for the DATs and Diego Smoke Test errands in diego-release (in flight)
• Re-run 10-cell performance experiment
• As a Diego developer, I would like a benchmark suite for the BBS's retrieval of DesiredLRP and ActualLRP data during LRP convergence
• A latency metric should not be emitted for requests to the BBS's events endpoints
• cloudfoundry-incubator/route-emitter #2: Include Route Service URL in Route Registration
• cloudfoundry-incubator/executor #14: fixes a typo, err -> info.Err
• Explore a minimal Diego deployment on AWS with continual stress to reproduce auction communication failures
• Update golang.org/x/crypto
• bump gomega in diego-release to include VerifyProtoRepresenting and RespondWithProto
• replace db/consul with locket
• remove garden-linux support blobs from diego-release
• remove dead code from runtime-schema

BOSH job changes

• etcd job removed from diego-release in favor of the same job from etcd-release.

BOSH property changes

• Add diego.stager.insecure_docker_registry_list: List of insecure Docker registries to allow access to when staging.
• Remove diego.stager.insecure_docker_registry.

Diego 0.1434.0

Changes from 0.1432.0 to 0.1434.0

• Depends on garden-linux-release v0.307.0.

Breaking changes from 0.1432.0

Buildpack Staging Response

• The staging response from Diego to CC has changed format to be better suited for the process-types work coming in the v3 CC API. Once the CF deployment has been updated to a version past commit ea88d1e, which will be in CF v219, CC and Diego will be able to communicate correctly during staging. Associated to CAPI story "the buildpack lifecycle should return all process types in the staging response".

Other significant changes

• Garden's Ping method should return an error if the btrfs graph path is not writable
• Diego should not set CF_STACK env var for running CF instances
• The BBS's etcd clients should avoid doing SSL handshakes with etcd unnecessarily
• BBS clients should avoid doing SSL handshakes with the BBS server unnecessarily
• cloudfoundry-incubator/windows_app_lifecycle #3: OutputMetadata conforms to Buildpack Staging Response protocol
• Update diego-release to golang 1.4.3
• Bump cloudfoundry-incubator/garden in diego-release
• The Diego BOSH manifest should configure garden to enable graph cleanup by default

BOSH job changes

None.

BOSH property changes

• Add parameters for tuning SSL connections between BBS clients and servers:
  • Properties:
    • diego.*.bbs.client_session_cache_size: Size of session ID cache for component's BBS client to maintain.
    • diego.*.bbs.max_idle_conns_per_host: Number of idle connections for each BBS client to maintain to BBS servers
  • Jobs:
    • auctioneer
    • converger
    • nsync
    • rep
    • route_emitter
    • ssh_proxy
    • stager
    • tps
• Add diego.bbs.etcd.client_session_cache_size: Size of session ID cache for BBS's etcd client to maintain.
• Add diego.bbs.etcd.max_idle_conns_per_host: Number of idle connections for BBS's etcd client to maintain to etcd.

Overrides of garden-linux-release property defaults

The spiff-based manifest-generation templates in diego-release provide values for the following properties in garden-linux-release:

• garden.enable_graph_cleanup: Default to true.
• garden.persistent_image_list: Default to ["/var/vcap/packages/rootfs_cflinuxfs2/rootfs"]

Values in the property-overrides.yml stub can override both of these defaults.

Diego 0.1433.0

Create final release 0.1433.0

Diego 0.1432.0

Changes from 0.1431.0 to 0.1432.0

• Depends on garden-linux-release v0.306.0.

Breaking changes from 0.1431.0

SSH Authentication to CF Instances

Associated to Diego story "The Diego SSH Proxy no longer accepts a user's access token as an SSH password for CF app instances".

Diego's SSH proxy no longer accepts a CF user's access token as a password for access to a CF app instance. It will instead accept only a one-time authorization code issued by UAA for its client. This client must also be registered with the UAA: for example, this client is registered for BOSH-lite deployments. As long as the name of the client is ssh-proxy, CC will advertise the correct client name in its /v2/info endpoint, and the Diego manifest-generation templates will flow the client secret to the SSH Proxy job.

For SSH access to CF app instances running on this release, we recommend you upgrade to version 0.2.0 or later of the Diego SSH plugin, or consult the diego-ssh repo for the current curl-based instructions to request a code from UAA.

Other significant changes

SSH

• The Diego SSH Proxy can receive an authorization code as the SSH password to access a CF app instance
• The SSH plugin provides a command to print a one-time authorization code issued for the SSH proxy client
• The SSH plugin establishes SSH connections to CF app instances by sending an authorization code as the SSH password
• As a CF user, when I establish a port-forwarding session with the SSH plugin, I expect it not to drop when going through a load balancer with an idle timeout
• Audit records for SSH access to CF instances should include which index was accessed

Performance

• BBS clients should avoid doing SSL handshakes with the BBS server unnecessarily (still in flight)
• The BBS's etcd clients should avoid doing SSL handshakes with etcd unnecessarily (still in flight)
• Change default route-emitter communication timeout to 30s

Misc

• Remove the Receptor
• Merge PRs for Routing info change
• operator should be able to verify that when mapping apps to a route already bound to a service instance for which the broker returned a route_service_url, CC sends updateDesiredLRP calls to Diego
• Merge PRs for CAPI backwards-incompatible change to Diego staging response
• the buildpack lifecycle should return all process types in the staging response
• The Diego BOSH manifest should include the cflinuxfs2 rootfs path in the garden.persistent_image_list property
• As a Diego operator, if a cell rep fails to start because it cannot ping Garden successfully, I can see a metric reporting how long the rep has been stalled
• Bump cflinuxfs2 stack to 1.9.0+
• Document how to secure BBS with mutual SSL auth in the diego-release README
• cloudfoundry-incubator/rep #3: Update client test to mach go 1.5 http errors
• cloudfoundry-incubator/executor #13: Improve leak detection in keyed lock test
• cloudfoundry/dropsonde #10: Added HasValue to FakeMetricSender

BOSH job changes

• Remove receptor job. The Lattice team has taken ownership of the receptor component and will be maintaining it from now on.

BOSH property changes

• Add diego.ssh_proxy.uaa_token_url: URL for the SSH proxy to use to request an access token from the UAA in exchange for its one-time auth code.
• Add diego.ssh_proxy.uaa_secret: Client secret for the SSH proxy to supply to UAA.
• Remove all properties under diego.receptor.

Diego 0.1431.0

Changes from 0.1430.0 to 0.1431.0

• Depends on garden-linux-release v0.305.0.

Breaking changes

Auction request payloads

Associated to Diego story "The auction should only send resources and identifiers back and forth". This may cause instance downtime during an upgrade from 0.1430.0 and earlier.

Mutual SSL Auth to BBS

Associated to Diego story "All communication with the BBS should be secured via mutually-authenticated SSL". By default, the BBS will now require mutual SSL authentication for access to its API endpoints. If this is enabled, components on an older release will be unable to communicate with the BBS when deploying an update, so cells may be unable to evacuate.

To configure the BBS with SSL correctly, it is easiest to use the scripts/generate-bbs-certs script to generate a CA certificate and key and certificates and keys for the BBS server and its clients. The contents of these certificates and client and server keys must then be included in the deployment manifest. If using the spiff-based manifest-generation tooling, these values can be included in the property-overrides.yml stub once and will flow to the BBS server and its clients.

Significant changes

• DesiredLRP data should be split across separate records
• As a BBS client, I can efficiently get frequently accessed data for all DesiredLRPs in a domain
• NSYNC's bulker should fetch the minimal set of DesiredLRP data
• Route-Emitter's bulk loop should fetch the minimal set of DesiredLRP data
• If a migration fails, BOSH aborts the deploy and I should be able to BOSH deploy the previously deployed release and recover.
• If no /version key is present in etcd, the BBS should not run any migrations
• As a Diego developer, I would like to run vizzini as an errand
• As a Diego operator, I can specify a set of decryption keys to use to decrypt data at rest, with the BBS migrating data to the designated active key in the set
• Diego etcd on bosh-lite should default to requiring ssl
• As a Diego operator, I can opt out of the SSH DATs that do not use the plugin
• vizzini test errand runs against BBS with mutual SSL auth enabled
• Provide vizzini job with BBS URL and local consul agent URL

BOSH job changes

• Added vizzini job to run the vizzini test suite as an errand.

BOSH property changes

• Added acceptance_tests.skip_ssh_without_plugin_tests: When true, skip SSH DATs that do not use the SSH plugin.
• Added properties for vizzini job:
  • vizzini.bbs.api_location: Address for vizzini to reach the BBS.
  • vizzini.routable_domain_suffix: Domain to use for vizzini to register routes during the test.
  • vizzini.nodes: Number of tests to run in parallel.
  • vizzini.verbose: Whether to log verbosely during the test run.
• Added BBS encryption properties:
  • diego.bbs.encryption_keys: List of label/passphrase pairs available to the BBS for encryption.
  • diego.bbs.active_key_label: Label of the encryption key to be used to encrypt the database.
• Added BBS mutual SSL auth properties:
  • Properties for BBS server job:
    • diego.bbs.require_ssl: whether the BBS requires SSL for communication.
    • diego.bbs.ca_cert: CA certificate used to sign BBS client and server SSL certificates.
    • diego.bbs.server_cert: SSL certificate that the BBS presents.
    • diego.bbs.server_key: Private key paired with the BBS's SSL certificate.
  • New BBS properties for client jobs:
    • Properties:
      • diego.*.bbs.ca_cert
      • diego.*.bbs.client_cert
      • diego.*.bbs.client_key
      • diego.*.bbs.require_ssl
    • Jobs:
      • auctioneer
      • converger
      • nsync
      • receptor
      • rep
      • route_emitter
      • ssh_proxy
      • stager
      • tps
      • vizzini
• Changed diego.*.bbs.api_url to diego.*.bbs.api_location for all jobs using the old property.
• Removed etcd communication properties from Diego core jobs:
  • Properties:
    • diego.*.etcd.machines
    • diego.*.etcd.ca_cert
    • diego.*.etcd.client_cert
    • diego.*.etcd.client_key
    • diego.*.etcd.require_ssl
  • Jobs:
    • auctioneer
    • converger
    • receptor
    • rep

Diego 0.1430.0

Version 0.1430.0 of Diego is recommended for use with CF v218.

Changes from 0.1428.0 to 0.1430.0

• Depends on garden-linux-release v0.305.0.

Configuration notes

• If upgrading from 0.1428.0 to 0.1430.0, we recommend you to deploy 0.1428.0 with the diego.bbs.serialization_format BOSH property set to proto. 0.1430.0 contains a BBS migration that encodes all the data in etcd as protobufs, which the first BBS server that receives the update will run. Setting this property to proto in advance guarantees that the other BBS servers will not accidentally write JSON-encoded records back into etcd before they also update to 0.1430.0. Note that unfortunately this property is not configurable via the manifest-generation templates in 0.1428.0, but it can be added directly to the properties section of the BOSH manifest.

Significant changes

• cloudfoundry-incubator/diego-release #72: ./scripts/update should fail fast when permission denied
• All CC-Bridge communication should happen directly with the BBS
• All Route-Emitter communication should happen directly with the BBS
• All SSH-Proxy communication should happen directly with the BBS
• All access to the BBS should go through one, master-elected, BBS server
• BBS server should emit metrics, remove the metrics server
• After a BOSH deploy, all data in the BBS should be stored in base64 encoded protobuf format
• If the Rep repeatedly fails to mark its ActualLRPs as EVACUATING it should fail to drain and the BOSH deploy should abort.
• Bump up the converger http timeout to one minute
• Never log environment variables and commands/arguments
• BBS Client should retry requests that fail because the BBS is migrating/lost the lock
• update cflinuxfs2 rootfs to 1.8.0+
• cloudfoundry-incubator/diego-ssh #5: Add -skipCertVerify to ssh-proxy
• The windows app lifecycle bundle should include a dummy diego-sshd executable
• provide user with helpful error message when they push a non-valid app

BOSH job changes

• Remove runtime_metrics_server job: the BBS server now emits Task and LRP metrics during convergence, and periodically emits etcd metrics.

BOSH property changes

• Add diego.nsync.bbs.api_url: Address for the Nsync processes to contact the BBS server.
• Add diego.route_emitter.bbs.api_url: Address for the Route-Emitter to contact the BBS server.
• Add diego.ssh_proxy.bbs.api_url: Address for the SSH-Proxy to contact the BBS server.
• Add diego.ssh_proxy.diego_credentials: Credentials to be used with the Diego authentication method.
• Add diego.tps.bbs.api_url: Address for the TPS processes to contact the BBS server.
• Remove diego.bbs.serialization_format.
• Remove diego.nsync.diego_api_url.
• Remove diego.route_emitter.diego_api_url.
• Remove diego.ssh_proxy.diego_api_url.
• Remove diego.tps.diego_api_url.

Diego 0.1429.0

Create final release 0.1429.0