ci: refactor docker ci script and enable docker job in presubmit #12662
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Signed-off-by: Lizan Zhou <[email protected]>
Signed-off-by: Lizan Zhou <[email protected]>
Signed-off-by: Lizan Zhou <[email protected]>
|
@lizan I'm not sure who best to assign this to; do you want to pick someone yourself after CI is clean? |
Signed-off-by: Lizan Zhou <[email protected]>
Signed-off-by: Lizan Zhou <[email protected]>
|
@htuch can you take a look? I refactored docker CI script and the VRP image build. |
|
@lizan have you tested the VRP build by following the VRP instructions (including the ones for local rebuild)? |
|
@htuch yes I tried |
htuch
approved these changes
Aug 18, 2020
stedsome
pushed a commit
to stedsome/envoy
that referenced
this pull request
Aug 22, 2020
Signed-off-by: Yifan Yang <[email protected]> the introduction of post processing script Signed-off-by: Yifan Yang <[email protected]> add the macos dependency installation Signed-off-by: Yifan Yang <[email protected]> install macos dependency Signed-off-by: Yifan Yang <[email protected]> comment out the slack functionality Signed-off-by: Yifan Yang <[email protected]> introduce flaky tests Signed-off-by: Yifan Yang <[email protected]> formatting the flaky tests Signed-off-by: Yifan Yang <[email protected]> commentout the format checks for debugging Signed-off-by: Yifan Yang <[email protected]> configuration Signed-off-by: Yifan Yang <[email protected]> configuration Signed-off-by: Yifan Yang <[email protected]> configuration Signed-off-by: Yifan Yang <[email protected]> configuration Signed-off-by: Yifan Yang <[email protected]> more configuration Signed-off-by: Yifan Yang <[email protected]> debugging Signed-off-by: Yifan Yang <[email protected]> install slack package for real Signed-off-by: Yifan Yang <[email protected]> add_condition Signed-off-by: Yifan Yang <[email protected]> try to install slack Signed-off-by: Yifan Yang <[email protected]> changed find path Signed-off-by: Yifan Yang <[email protected]> trying to locate the test result folders Signed-off-by: Yifan Yang <[email protected]> trying to find the test folders Signed-off-by: Yifan Yang <[email protected]> trying to find the test folders Signed-off-by: Yifan Yang <[email protected]> trying to find the test folders Signed-off-by: Yifan Yang <[email protected]> trying to find the test folders Signed-off-by: Yifan Yang <[email protected]> trying to find the test folders Signed-off-by: Yifan Yang <[email protected]> finding test files Signed-off-by: Yifan Yang <[email protected]> finding test files Signed-off-by: Yifan Yang <[email protected]> finding test files Signed-off-by: Yifan Yang <[email protected]> finding test files Signed-off-by: Yifan Yang <[email protected]> finding test files Signed-off-by: Yifan Yang <[email protected]> finding test files Signed-off-by: Yifan Yang <[email protected]> find the test files Signed-off-by: Yifan Yang <[email protected]> find the test files Signed-off-by: Yifan Yang <[email protected]> find the test files Signed-off-by: Yifan Yang <[email protected]> find the test files Signed-off-by: Yifan Yang <[email protected]> find the test files Signed-off-by: Yifan Yang <[email protected]> find the test files Signed-off-by: Yifan Yang <[email protected]> find the test files Signed-off-by: Yifan Yang <[email protected]> find the test files Signed-off-by: Yifan Yang <[email protected]> find the test files Signed-off-by: Yifan Yang <[email protected]> find the test files Signed-off-by: Yifan Yang <[email protected]> find the test files Signed-off-by: Yifan Yang <[email protected]> python dependencies Signed-off-by: Yifan Yang <[email protected]> python dependencies Signed-off-by: Yifan Yang <[email protected]> python dependencies Signed-off-by: Yifan Yang <[email protected]> python dependencies Signed-off-by: Yifan Yang <[email protected]> python dependencies Signed-off-by: Yifan Yang <[email protected]> python dependencies Signed-off-by: Yifan Yang <[email protected]> python dependencies Signed-off-by: Yifan Yang <[email protected]> testing coverage builds Signed-off-by: Yifan Yang <[email protected]> testing coverage builds Signed-off-by: Yifan Yang <[email protected]> try to figure out coverage builds Signed-off-by: Yifan Yang <[email protected]> comment out slack Signed-off-by: Yifan Yang <[email protected]> testing coverage build Signed-off-by: Yifan Yang <[email protected]> testing coverage build Signed-off-by: Yifan Yang <[email protected]> trying slack features Signed-off-by: Yifan Yang <[email protected]> more formatting Signed-off-by: Yifan Yang <[email protected]> slack Signed-off-by: Yifan Yang <[email protected]> adding CI_Target to output msg Signed-off-by: Yifan Yang <[email protected]> force a run Signed-off-by: Yifan Yang <[email protected]> fuzz: added fuzz test for listener filter tls_inspector (envoyproxy#12617) Created tls_inspector_corpus and populated with testcases (valid and invalid client hellos) Risk Level: Low Testing: increased function coverage of tls_inspector.cc to 100.0% and line coverage to 87.3% after running fuzzer (covers all parse states except errors related to socket read failure). Docs Changes: N/A Release Notes: N/A Signed-off-by: Arthur Yan <[email protected]> scoped_rds_integration_test migrate from api v2 to api v3. (envoyproxy#12633) Migrate the integration test of scoped rds from api v2 to api v3. Fix a bug in scoped_rds.cc: ScopedRdsConfigSubscription should use the resource version of srds, not the resource version of rds. Risk Level: Low Signed-off-by: chaoqinli <[email protected]> add 'explicit' restriction. (envoyproxy#12643) Commit Message: The intent, when providing Stats::Utility::counterFromElements, is that dynamic segments should be easy to construct, but still searchable. We should be trying to avoid dynamic segments whenever possible, so having them implicitly created from string data is not idea. Additional Description: Risk Level: none for the repo, but possibly will require trivial edits outside the repo Testing: //test/... Docs Changes: n/a Release Notes: n/a Signed-off-by: Joshua Marantz <[email protected]> [fuzz]added an input check in writefilter fuzzer and added test cases (envoyproxy#12628) Added a handle for nullptr in HeaderPercentageProvider::percentage to avoid crash in mongo_proxy. Added many unit test cases into corpus so that the coverage can be improved. All those filters' coverage was increased by 20%-40%. Signed-off-by: jianwen <[email protected]> test: fix http_timeout_integration_test flake (envoyproxy#12654) Fixes envoyproxy#12653 Signed-off-by: Matt Klein <[email protected]> logger: support log control in admin interface and command line option for Fancy Logger (envoyproxy#12369) Add log control (list and modify log level) in admin interface for Fancy Logger, a new fine-grained logger for Envoy, and provide command line option --enable-fine-grain-logging for developers. Additional Description: A doc of overview is provided here: source/docs/fancy_logger.md. Risk Level: Medium Testing: Unit tests. Docs Changes: Added a new option --enable-fine-grain-logging and doc it. Release Notes: Added to current.rst. Signed-off-by: Jinhui Song <[email protected]> Decreased the flakiness of Watchdog tests running real time system. (envoyproxy#12659) Signed-off-by: Kevin Baichoo <[email protected]> test: fix ext auth flake (envoyproxy#12660) Fixes envoyproxy#12657 Signed-off-by: Matt Klein <[email protected]> test: deflake timer test, not completely (envoyproxy#12642) Signed-off-by: Lizan Zhou <[email protected]> test: fix ProtocolIntegrationTest.LargeRequestMethod flake (envoyproxy#12661) Fixes envoyproxy#12484 Signed-off-by: Matt Klein <[email protected]> Fix proto_sync.py (envoyproxy#12434) Fix path in proto_sync.py generated comments and regenerate. Signed-off-by: wzshiming <[email protected]> udp: Add some log when session is deleted (envoyproxy#12669) It is very helpful for debugging. Signed-off-by: DongRyeol Cha <[email protected]> DNS filter: set default resolver timeout (envoyproxy#12293) Fix an ASAN failure in certain env. Risk Level: low Testing: n/a Docs Changes: n/a Release Notes: n/a Signed-off-by: Yuchen Dai <[email protected]> http: remove deprecated envoy.reloadable_features.connection_header_sanitization runtime guard (envoyproxy#12500) http: Remove deprecated envoy.reloadable_features.connection_header_sanitization runtime guard Signed-off-by: Alvin Baptiste [email protected] Commit Message: http: remove deprecated runtime guard for connection header sanitization Risk Level: Low Testing: bazel test //test/... Release Notes: Added Fixes envoyproxy#11933 Removed: envoy.reloadable_features.connection_header_sanitization Signed-off-by: Alvin Baptiste <[email protected]> Fix broken codeb lock style (envoyproxy#12667) Fix broken code block style in docs Signed-off-by: Takao Shibata <[email protected]> lua: Manage imported public keys in stream handle (envoyproxy#12664) This patch manages the imported public keys in the stream-handle object instead of "exposing" it as pointer through lua_pushlightuserdata while preserving the current Lua APIs. Signed-off-by: Dhi Aurrahman <[email protected]> lua API: add base64Escape function to stream handle (envoyproxy#12552) This makes it easy for Lua filters to base64 escape strings without needing to provide their own base64 helper. Signed-off-by: Michael Puncel <[email protected]> [Windows] Fixes Udp listener tests (envoyproxy#12635) Fixes UDP listener tests on Windows by modernizing iovecToWSABUF and msghdrToWSAMSG from pointer arithmetic to c++ and makes message.msg_controllen the proper length. Signed-off-by: Sotiris Nanopoulos <[email protected]> api: deprecate the node.listening_addresses field (envoyproxy#12691) This was added for gRPC server support, but we've decided to use resource names instead to explicitly request the listeners we want by name. This is more in-line with the new naming scheme described in the "xDS Transport Next Steps" design. Signed-off-by: Mark D. Roth <[email protected]> Fix broken reST style (envoyproxy#12668) Signed-off-by: Takao Shibata <[email protected]> Windows: Fix filesystem subscription impl tests (envoyproxy#12597) Windows: Fix filesystem subscription impl tests Replace dispatcher and watcher with mocks and orchestrate the test harness to capture filesystem subscription impl file event callback and invoke it directly. Test no longer relies on watching real filesystem events and avoids timing discrepancies causing flakiness. Additional Description: N/A Risk Level: Low Testing: Modifies unit tests, tested locally on Windows Docs Changes: N/A Release Notes: N/A Signed-off-by: Sunjay Bhatia <[email protected]> Co-authored-by: William A Rowe Jr <[email protected]> Fixes stack overflow in http inspector test (envoyproxy#12577) Fixes stack overflow exception in HttpInspectorTest.Http1WithLargeRequestLine and makes the test faster. Additional Description: While I was working on level vs edge based events I observed that the test is causing a stack overflow on Windows/MSVC. The testlist http_inspector_test now passes on Windows but it should not because it relies on Event::FileTriggerType::Edge which are not supported. This is why I did not enable it for the CI. Risk Level: N/A Testing: N/A Docs Changes: N/A Release Notes: N/A Signed-off-by: Sotiris Nanopoulos <[email protected]> docs: external dependency dashboard. (envoyproxy#12639) This patch introduces a set of automatically generated tables (based on repository_locations.bzl) that enumerate the external dependencies that feature on Envoy's data/control planes, test, build, etc. Version and CPE information is currently included. In the future, we will also have last updated, distinguish core vs. extensions and populate with external dependency process maturity information. Part of envoyproxy#10471. This is essentially providing a programmatic variant of envoyproxy#10471 (comment). Future enhancements are tracked at envoyproxy#12673. Signed-off-by: Harvey Tuch <[email protected]> [fuzz]expand readfilter_fuzzer to cover mongo_proxy and mysql_proxy (envoyproxy#12612) Added coverage for mongo_proxy and mysql_proxy Added test cases(corpus) for them. Signed-off-by: jianwen <[email protected]> xds: allow updating listener back to original state (envoyproxy#12645) since addOrUpdateListenerInternal returns early in the case of a duplicate active/warming listener being added, it means you cannot update a listener back to its original state after updating it to a warming state consider the following sequence of actions: * add listener_0 referencing route_config_0, make it active * update listener_0 referencing route_config_1, keep it warming (Envoy keeps the original listener_0 active until the new warms) * update listener_0 back to route_config_0, which should remove the warming listener and return Envoy to its initial state, so that a future addition of route_config_1 won't cause the listener to change to that state, but right now it doesn't make a small change in listener_manager_impl.cc to allow this to happen update xds_verifier.cc and add a unit test for this case Risk Level: Medium Testing: Passes existing fuzz corpora and a few minutes on libfuzzer, passes ads_integration_test and lds_api_test Docs Changes: N/A Release Notes: N/A Fixes: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=24888 Signed-off-by: Sam Flattery <[email protected]> ci: refactor docker ci script and enable docker job in presubmit (envoyproxy#12662) Signed-off-by: Lizan Zhou <[email protected]> [test] using reserve and commit in watermark_buffer overflow high value test (envoyproxy#12494) Improving the test by avoiding actual copying of bytes, but just reserving and committing slices. Signed-off-by: Adi Suissa-Peleg <[email protected]> Greenlight and reclassify //test/... with the current progress for Windows (envoyproxy#12695) Co-authored-by: William A Rowe Jr <[email protected]> Co-authored-by: Sunjay Bhatia <[email protected]> Signed-off-by: William A Rowe Jr <[email protected]> Signed-off-by: Sunjay Bhatia <[email protected]> hds: group endpoint health response by cluster and locality (envoyproxy#12452) Currently, the health discovery service takes a specifier with information about which endpoints to perform health checks on, which already supports information about cluster structure and locality information. However, when when forming a response holding endpoint health information, all notion of cluster or locality is dropped and endpoint information is stored in a flat list. This is problematic when there are several endpoints with the same address or port but with a different locality, cluster, or path. This PR uses the previous proto change in Issue envoyproxy#12153 which added support for grouping endpoint health information by their respective cluster and locality. Risk Level: Low Testing: Added a unit test in test/common/upstream/hds_test, which sends a specifier to HdsDelegate with several clusters, localties, and endpoints. It then verifies that the response holds the same structure. Existing integration tests were also changed to check for the new proto structure, specifically ones that already group several endpoints by differing clusters or localities. Signed-off-by: Drew S. Ortega <[email protected]> network: socket and address build cleanup (envoyproxy#12710) - split socket interface from socket - add default socket interface library - move io handle to default socket interface library from address Signed-off-by: Florin Coras <[email protected]> thrift: envoy_cc_test -> envoy_extension_cc_test (envoyproxy#12697) Signed-off-by: Roelof DuToit <[email protected]> test: Add test socket interface that allows overriding IoHandle behavior (envoyproxy#12528) Add test socket interface that allows overriding IoHandle behavior of accepted sockets. Change flood tests to use exact frame counts needed for flooding. Fix DATA frame flood test. Signed-off-by: Yan Avlasov <[email protected]> fix main branch merge issue (envoyproxy#12722) Signed-off-by: Florin Coras <[email protected]> ci: fix VRP image push (envoyproxy#12715) Signed-off-by: Lizan Zhou <[email protected]> cosmetic changes Signed-off-by: Yifan Yang <[email protected]> add a flaky test in macOS build Signed-off-by: Yifan Yang <[email protected]> add a flaky test in macOS build Signed-off-by: Yifan Yang <[email protected]> changed the flaky test Signed-off-by: Yifan Yang <[email protected]> changed the flaky test Signed-off-by: Yifan Yang <[email protected]> have a wrapper script Signed-off-by: Yifan Yang <[email protected]> trying necessary dependencies Signed-off-by: Yifan Yang <[email protected]> trying necessary dependencies Signed-off-by: Yifan Yang <[email protected]> testing python dependencies Signed-off-by: Yifan Yang <[email protected]> add a shell script wrapper and use python_venv Signed-off-by: Yifan Yang <[email protected]> moved functionality into do_ci.sh and added the hyperlink Signed-off-by: Yifan Yang <[email protected]> moved functionality into do_ci.sh and added the hyperlink Signed-off-by: Yifan Yang <[email protected]> put it out of ci_do.sh Signed-off-by: Yifan Yang <[email protected]> put it out of ci_do.sh Signed-off-by: Yifan Yang <[email protected]> put it out of ci_do.sh Signed-off-by: Yifan Yang <[email protected]> ci_do.sh Signed-off-by: Yifan Yang <[email protected]> ci_do.sh Signed-off-by: Yifan Yang <[email protected]> ci_do.sh Signed-off-by: Yifan Yang <[email protected]> more setup Signed-off-by: Yifan Yang <[email protected]> more setup Signed-off-by: Yifan Yang <[email protected]> more setup Signed-off-by: Yifan Yang <[email protected]> more setup Signed-off-by: Yifan Yang <[email protected]> more setup Signed-off-by: Yifan Yang <[email protected]> more setup Signed-off-by: Yifan Yang <[email protected]> more setup Signed-off-by: Yifan Yang <[email protected]> test hyperlink Signed-off-by: Yifan Yang <[email protected]> installing with py3 Signed-off-by: Yifan Yang <[email protected]> making it work Signed-off-by: Yifan Yang <[email protected]> pass the CI-target in Signed-off-by: Yifan Yang <[email protected]> changing the search path Signed-off-by: Yifan Yang <[email protected]> changing the search path Signed-off-by: Yifan Yang <[email protected]> add a set to keep track of seen problem Signed-off-by: Yifan Yang <[email protected]> fix of something stupid Signed-off-by: Yifan Yang <[email protected]> add some comments and retry arm64 Signed-off-by: Yifan Yang <[email protected]> getting it ready for shipping Signed-off-by: Yifan Yang <[email protected]> format Signed-off-by: Yifan Yang <[email protected]> more format Signed-off-by: Yifan Yang <[email protected]> format Signed-off-by: Yifan Yang <[email protected]> try again with arm64 Signed-off-by: Yifan Yang <[email protected]> testing arm build Signed-off-by: Yifan Yang <[email protected]> update requirement for arm Signed-off-by: Yifan Yang <[email protected]> trying out arm dependencies Signed-off-by: Yifan Yang <[email protected]> debugging Signed-off-by: Yifan Yang <[email protected]> opting out of arm arch for now Signed-off-by: Yifan Yang <[email protected]> cleanup Signed-off-by: Yifan Yang <[email protected]> commentout formatting for faster builds Signed-off-by: Yifan Yang <[email protected]> upgrading setuptools Signed-off-by: Yifan Yang <[email protected]> dependency Signed-off-by: Yifan Yang <[email protected]> dependency Signed-off-by: Yifan Yang <[email protected]> dependency Signed-off-by: Yifan Yang <[email protected]> dependency Signed-off-by: Yifan Yang <[email protected]> dependency Signed-off-by: Yifan Yang <[email protected]> dependency Signed-off-by: Yifan Yang <[email protected]> dependency Signed-off-by: Yifan Yang <[email protected]> dependency Signed-off-by: Yifan Yang <[email protected]> sanity check Signed-off-by: Yifan Yang <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Signed-off-by: Lizan Zhou [email protected]