datastore: fix to backoff global / race condition now no longer exponentially backs off

[sgp]

Client

Cloud Datastore

Environment

All Environments

Expected behavior

All transaction-related retries, including creating and rolling back, should utilize exponential backoff.

Actual behavior

A local gax.OnCode call is made inside each try, which ensures that the pause between tries will always be the minimum 20ms specified by txnBackoff (+/- the random jitter), regardless of whether it's the first, second, or nth retry.

Additional context

This was introduced in #10489 (1.1.18) which fixed a bigger bug introduced in #10349 (1.17.1) that actually bit us in production. In Datastore 1.17.1, the retryers were global, and thus eventually, if you had enough retries between transactions, the first retry would eventually be the maximum time. For this reason, I recommend that Google consider retracting version 1.17.1 of the Cloud Datastore Client entirely.

[sgp]

Is it too late to get this in the 1.20.0 component release for datastore?

[bhshkh]

Will include in 1.20.0