Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

feat(bigquery/storage/managedwriter): enable field name indirection #6247

Merged
merged 30 commits into from
Oct 4, 2022
Merged

feat(bigquery/storage/managedwriter): enable field name indirection #6247

merged 30 commits into from
Oct 4, 2022

Conversation

shollyman
Copy link
Contributor

@shollyman shollyman commented Jun 23, 2022 •
edited
Loading

This PR wires in validation testing for extension-based naming indirection,
and adds support for rich names to the adapt subpackage.

This feature is still in preview, but this will enable use for customers enrolled in the preview.

Because this change effectively allows special characters in field names, this PR also
augments testing utilities that emit SQL to escape said identifiers.

@shollyman
feat(bigquery/storage/managedwriter): expose preview annotation
2b8c0fd 
This PR adds the preview annotation within an internal package
to enable early development without exposing the surface to
end users.

It doesn't include any modifications to the behavior
@product-auto-label product-auto-label bot added size: m Pull request size is medium. api: bigquery Issues related to the BigQuery API. labels Jun 23, 2022
@shollyman shollyman requested a review from yirutang June 23, 2022 18:35
yirutang
yirutang reviewed Jun 23, 2022
View reviewed changes
Copy link
Contributor

@yirutang yirutang left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why is it here? Should it be in some common place?

yirutang
yirutang reviewed Jun 23, 2022
View reviewed changes
Copy link
Contributor

@yirutang yirutang left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So this is "THE" staging place?

@shollyman
Copy link
Contributor Author

So this is "THE" staging place?

This is the place for golang. We'll do something similar for java. See b/235906168 for more info.

@product-auto-label product-auto-label bot added size: l Pull request size is large. and removed size: m Pull request size is medium. labels Jul 15, 2022
@product-auto-label product-auto-label bot added the stale: old Pull request is old and needs attention. label Jul 24, 2022
@product-auto-label product-auto-label bot added stale: extraold Pull request is critically old and needs prioritization. and removed stale: old Pull request is old and needs attention. labels Aug 23, 2022
@shollyman
Copy link
Contributor Author

Not an expert in golang, but the base64 library here is similar to java's. With stdEncoder the encoded field name will contain + and /

Good spot, didn't think to doublecheck the allowed characters. Decided to stick with standard encoding, and redact the problematic characters (+,/,=).

@shollyman shollyman changed the title feat(bigquery/storage/managedwriter): expose preview annotation feat(bigquery/storage/managedwriter): enable field name indirection Sep 20, 2022
@shollyman shollyman marked this pull request as ready for review September 20, 2022 16:53
@shollyman shollyman requested a review from a team September 20, 2022 16:53
@shollyman shollyman requested a review from a team as a code owner September 20, 2022 16:53
@shollyman shollyman added the automerge Merge the pull request once unit tests and other checks pass. label Oct 4, 2022
@gcf-merge-on-green gcf-merge-on-green bot merged commit 1969273 into googleapis:main Oct 4, 2022
@gcf-merge-on-green gcf-merge-on-green bot removed the automerge Merge the pull request once unit tests and other checks pass. label Oct 4, 2022
@shollyman shollyman deleted the annotation-preview branch October 4, 2022 19:12
@release-please release-please bot mentioned this pull request Oct 4, 2022
Merged
gcf-merge-on-green bot pushed a commit that referenced this pull request Oct 24, 2022
@release-please
chore(main): release bigquery 1.43.0 (#6730)
716ea88 
🤖 I have created a release *beep* *boop*
---


## [1.43.0](https://togithub.com/googleapis/google-cloud-go/compare/bigquery/v1.42.0...bigquery/v1.43.0) (2022-10-24)


### Features

* **bigquery/analyticshub:** rename nodejs analyticshub library package name ([52dddd1](https://togithub.com/googleapis/google-cloud-go/commit/52dddd1ed89fbe77e1859311c3b993a77a82bfc7))
* **bigquery/storage/managedwriter:** Enable field name indirection ([#6247](https://togithub.com/googleapis/google-cloud-go/issues/6247)) ([1969273](https://togithub.com/googleapis/google-cloud-go/commit/19692735b0fbafa176d0315bda923528e1eedf6d))
* **bigquery/storage/managedwriter:** Retry on FailedPrecondition ([#6761](https://togithub.com/googleapis/google-cloud-go/issues/6761)) ([d1a444d](https://togithub.com/googleapis/google-cloud-go/commit/d1a444d769c9578b586bef608d343b4b0abd3658))
* **bigquery/storage/managedwriter:** Support append retries ([#6695](https://togithub.com/googleapis/google-cloud-go/issues/6695)) ([6ae9c67](https://togithub.com/googleapis/google-cloud-go/commit/6ae9c670a11d80b34872cb05fda933303b81851d))
* **bigquery/storage/managedwriter:** Switch to opt-in retry ([#6765](https://togithub.com/googleapis/google-cloud-go/issues/6765)) ([a3e97a6](https://togithub.com/googleapis/google-cloud-go/commit/a3e97a6f15ad1989ef815b9bd5838192f9f226f1))
* **bigquery:** Add remote function options to routine metadata ([#6702](https://togithub.com/googleapis/google-cloud-go/issues/6702)) ([d9a437d](https://togithub.com/googleapis/google-cloud-go/commit/d9a437de75a5f5151cd000d8f9a6b7fc567d8551))


### Bug Fixes

* **bigquery/storage/managedwriter:** Address possible resource leak ([#6775](https://togithub.com/googleapis/google-cloud-go/issues/6775)) ([979440b](https://togithub.com/googleapis/google-cloud-go/commit/979440b43573c1cfd744d3f63d0c633c7319ad46))
* **bigquery:** Avoid stack overflow on query param with recursive types ([#6890](https://togithub.com/googleapis/google-cloud-go/issues/6890)) ([854ccfc](https://togithub.com/googleapis/google-cloud-go/commit/854ccfca259d747759d283fc0d0053893f3c8f8d)), refs [#6884](https://togithub.com/googleapis/google-cloud-go/issues/6884)
* **bigquery:** Bq connection auth scopes ([#6752](https://togithub.com/googleapis/google-cloud-go/issues/6752)) ([8e09288](https://togithub.com/googleapis/google-cloud-go/commit/8e09288185f721d90288c3aa873980fc44b98613)), refs [#6744](https://togithub.com/googleapis/google-cloud-go/issues/6744)


### Documentation

* **bigquery/storage/managedwriter/adapt:** Typo in error string ([#6729](https://togithub.com/googleapis/google-cloud-go/issues/6729)) ([bb26153](https://togithub.com/googleapis/google-cloud-go/commit/bb26153d38475cd9784edbf241df84c368f5a166))
* **bigquery/storage/managedwriter:** Add retry info to package docs ([#6803](https://togithub.com/googleapis/google-cloud-go/issues/6803)) ([81e52e5](https://togithub.com/googleapis/google-cloud-go/commit/81e52e59dcf3c4a44108e039fb0e3a0e2ce8284f))

---
This PR was generated with [Release Please](https://togithub.com/googleapis/release-please). See [documentation](https://togithub.com/googleapis/release-please#release-please).
kimihrr pushed a commit to kimihrr/google-cloud-go-kimia that referenced this pull request Oct 25, 2022
@release-please @kimihrr
chore(main): release bigquery 1.43.0 (googleapis#6730)
b854c65 
🤖 I have created a release *beep* *boop*
---


## [1.43.0](https://togithub.com/googleapis/google-cloud-go/compare/bigquery/v1.42.0...bigquery/v1.43.0) (2022-10-24)


### Features

* **bigquery/analyticshub:** rename nodejs analyticshub library package name ([52dddd1](https://togithub.com/googleapis/google-cloud-go/commit/52dddd1ed89fbe77e1859311c3b993a77a82bfc7))
* **bigquery/storage/managedwriter:** Enable field name indirection ([googleapis#6247](https://togithub.com/googleapis/google-cloud-go/issues/6247)) ([1969273](https://togithub.com/googleapis/google-cloud-go/commit/19692735b0fbafa176d0315bda923528e1eedf6d))
* **bigquery/storage/managedwriter:** Retry on FailedPrecondition ([googleapis#6761](https://togithub.com/googleapis/google-cloud-go/issues/6761)) ([d1a444d](https://togithub.com/googleapis/google-cloud-go/commit/d1a444d769c9578b586bef608d343b4b0abd3658))
* **bigquery/storage/managedwriter:** Support append retries ([googleapis#6695](https://togithub.com/googleapis/google-cloud-go/issues/6695)) ([6ae9c67](https://togithub.com/googleapis/google-cloud-go/commit/6ae9c670a11d80b34872cb05fda933303b81851d))
* **bigquery/storage/managedwriter:** Switch to opt-in retry ([googleapis#6765](https://togithub.com/googleapis/google-cloud-go/issues/6765)) ([a3e97a6](https://togithub.com/googleapis/google-cloud-go/commit/a3e97a6f15ad1989ef815b9bd5838192f9f226f1))
* **bigquery:** Add remote function options to routine metadata ([googleapis#6702](https://togithub.com/googleapis/google-cloud-go/issues/6702)) ([d9a437d](https://togithub.com/googleapis/google-cloud-go/commit/d9a437de75a5f5151cd000d8f9a6b7fc567d8551))


### Bug Fixes

* **bigquery/storage/managedwriter:** Address possible resource leak ([googleapis#6775](https://togithub.com/googleapis/google-cloud-go/issues/6775)) ([979440b](https://togithub.com/googleapis/google-cloud-go/commit/979440b43573c1cfd744d3f63d0c633c7319ad46))
* **bigquery:** Avoid stack overflow on query param with recursive types ([googleapis#6890](https://togithub.com/googleapis/google-cloud-go/issues/6890)) ([854ccfc](https://togithub.com/googleapis/google-cloud-go/commit/854ccfca259d747759d283fc0d0053893f3c8f8d)), refs [googleapis#6884](https://togithub.com/googleapis/google-cloud-go/issues/6884)
* **bigquery:** Bq connection auth scopes ([googleapis#6752](https://togithub.com/googleapis/google-cloud-go/issues/6752)) ([8e09288](https://togithub.com/googleapis/google-cloud-go/commit/8e09288185f721d90288c3aa873980fc44b98613)), refs [googleapis#6744](https://togithub.com/googleapis/google-cloud-go/issues/6744)


### Documentation

* **bigquery/storage/managedwriter/adapt:** Typo in error string ([googleapis#6729](https://togithub.com/googleapis/google-cloud-go/issues/6729)) ([bb26153](https://togithub.com/googleapis/google-cloud-go/commit/bb26153d38475cd9784edbf241df84c368f5a166))
* **bigquery/storage/managedwriter:** Add retry info to package docs ([googleapis#6803](https://togithub.com/googleapis/google-cloud-go/issues/6803)) ([81e52e5](https://togithub.com/googleapis/google-cloud-go/commit/81e52e59dcf3c4a44108e039fb0e3a0e2ce8284f))

---
This PR was generated with [Release Please](https://togithub.com/googleapis/release-please). See [documentation](https://togithub.com/googleapis/release-please#release-please).
This was referenced Nov 1, 2022
Closed
Closed
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@tracyz-g tracyz-g tracyz-g approved these changes

@yirutang yirutang yirutang approved these changes

@alvarowolfx alvarowolfx Awaiting requested review from alvarowolfx

Assignees
No one assigned
Labels
api: bigquery Issues related to the BigQuery API. size: l Pull request size is large. stale: extraold Pull request is critically old and needs prioritization.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@shollyman @yirutang @tracyz-g