transformer.proto
path mgmt/v1alpha1/transformer.proto
package mgmt.v1alpha1
Messages
CreateUserDefinedTransformerRequest
Name | Type | Description |
---|---|---|
account_id | string | |
name | string | |
description | string | |
type | string | @deprecated |
source | TransformerSource | |
transformer_config | TransformerConfig |
CreateUserDefinedTransformerResponse
Name | Type | Description |
---|---|---|
transformer | UserDefinedTransformer |
DeleteUserDefinedTransformerRequest
Name | Type | Description |
---|---|---|
transformer_id | string |
DeleteUserDefinedTransformerResponse
Name | Type | Description |
---|
GenerateBool
Name | Type | Description |
---|
GenerateBusinessName
Name | Type | Description |
---|
GenerateCardNumber
Name | Type | Description |
---|---|---|
valid_luhn | optional bool |
GenerateCategorical
Name | Type | Description |
---|---|---|
categories | optional string |
GenerateCity
Name | Type | Description |
---|
GenerateCountry
Name | Type | Description |
---|---|---|
generate_full_name | optional bool | An option to return the full country name of the randomly selected country or return the default of a 2-letter country code. |
GenerateDefault
Name | Type | Description |
---|
GenerateE164PhoneNumber
Name | Type | Description |
---|---|---|
min | optional int64 | |
max | optional int64 |
GenerateEmail
Name | Type | Description |
---|---|---|
email_type | optional GenerateEmailType | Optionally specify the type of email to generate. The types specified determine the contents on the left side of the @. |
GenerateFirstName
Name | Type | Description |
---|
GenerateFloat64
Name | Type | Description |
---|---|---|
randomize_sign | optional bool | |
min | optional double | |
max | optional double | |
precision | optional int64 |
GenerateFullAddress
Name | Type | Description |
---|
GenerateFullName
Name | Type | Description |
---|
GenerateGender
Name | Type | Description |
---|---|---|
abbreviate | optional bool |
GenerateInt64
Name | Type | Description |
---|---|---|
randomize_sign | optional bool | |
min | optional int64 | |
max | optional int64 |
GenerateInt64PhoneNumber
Name | Type | Description |
---|
GenerateIpAddress
Name | Type | Description |
---|---|---|
ip_type | optional GenerateIpAddressType |
GenerateJavascript
Name | Type | Description |
---|---|---|
code | string |
GenerateLastName
Name | Type | Description |
---|
GenerateSSN
Name | Type | Description |
---|
GenerateSha256Hash
Name | Type | Description |
---|
GenerateState
Name | Type | Description |
---|---|---|
generate_full_name | optional bool | An option to return the full state name of the randomly selected state or return the default of a 2-letter state code. |
GenerateStreetAddress
Name | Type | Description |
---|
GenerateString
Name | Type | Description |
---|---|---|
min | optional int64 | |
max | optional int64 |
GenerateStringPhoneNumber
Name | Type | Description |
---|---|---|
min | optional int64 | |
max | optional int64 |
GenerateUnixTimestamp
Name | Type | Description |
---|
GenerateUsername
Name | Type | Description |
---|
GenerateUtcTimestamp
Name | Type | Description |
---|
GenerateUuid
Name | Type | Description |
---|---|---|
include_hyphens | optional bool |
GenerateZipcode
Name | Type | Description |
---|
GetSystemTransformerBySourceRequest
Name | Type | Description |
---|---|---|
source | TransformerSource |
GetSystemTransformerBySourceResponse
Name | Type | Description |
---|---|---|
transformer | SystemTransformer |
GetSystemTransformersRequest
Name | Type | Description |
---|
GetSystemTransformersResponse
Name | Type | Description |
---|---|---|
transformers | repeated SystemTransformer |
GetTransformPiiEntitiesRequest
Name | Type | Description |
---|---|---|
account_id | string | The unique identifier of the account to return entities for. |
GetTransformPiiEntitiesResponse
Name | Type | Description |
---|---|---|
entities | repeated string | The list of built-in entities that are enabled for the provided account id. |
GetUserDefinedTransformerByIdRequest
Name | Type | Description |
---|---|---|
transformer_id | string |
GetUserDefinedTransformerByIdResponse
Name | Type | Description |
---|---|---|
transformer | UserDefinedTransformer |
GetUserDefinedTransformersRequest
Name | Type | Description |
---|---|---|
account_id | string |
GetUserDefinedTransformersResponse
Name | Type | Description |
---|---|---|
transformers | repeated UserDefinedTransformer |
IsTransformerNameAvailableRequest
Name | Type | Description |
---|---|---|
account_id | string | |
transformer_name | string |
IsTransformerNameAvailableResponse
Name | Type | Description |
---|---|---|
is_available | bool |
Null
Name | Type | Description |
---|
Passthrough
Name | Type | Description |
---|
PiiAnonymizer
Name | Type | Description |
---|---|---|
replace | PiiAnonymizer.Replace | Configures the anonymizer to replace the identified PII with a specific value |
redact | PiiAnonymizer.Redact | Configures the anonymizer to completely remove the identified PII |
mask | PiiAnonymizer.Mask | Configures the anonymizer to mask the identified PII |
hash | PiiAnonymizer.Hash | Configures the anonymizer to hash the identified PII |
PiiAnonymizer.Hash
Name | Type | Description |
---|---|---|
algo | optional PiiAnonymizer.Hash.HashType | The hashing algorithm to use. |
PiiAnonymizer.Mask
Name | Type | Description |
---|---|---|
masking_char | optional string | Provide a character that will be used for the replacement. |
chars_to_mask | optional int32 | How many characters to mask. |
from_end | optional bool | Whether to mask the PII from start or end |
PiiAnonymizer.Redact
Name | Type | Description |
---|
PiiAnonymizer.Replace
Name | Type | Description |
---|---|---|
value | optional string | The value to replace. If not provided, a template token of the anonymizer is used. |
PiiDenyRecognizer
Name | Type | Description |
---|---|---|
name | string | Friendly name of this entity |
deny_words | repeated string | List of words that will be treated as PII. |
SystemTransformer
Name | Type | Description |
---|---|---|
name | string | |
description | string | |
data_type | TransformerDataType | |
source | TransformerSource | |
config | TransformerConfig | |
data_types | repeated TransformerDataType | |
supported_job_types | repeated SupportedJobType |
TransformCharacterScramble
Name | Type | Description |
---|---|---|
user_provided_regex | optional string | a user provided regular expression that they wish to validate if it compiles in GO |
TransformE164PhoneNumber
Name | Type | Description |
---|---|---|
preserve_length | optional bool |
TransformEmail
Name | Type | Description |
---|---|---|
preserve_domain | optional bool | Whether or not to preserve the original domain, barring what has been specified in the excluded_domains property. |
preserve_length | optional bool | Whether or not to preserve the original length of the email. This causes the transformed email to retain the original length. |
excluded_domains | repeated string | A lsit of email domains that should be excluded. This changes based on the preserve_domain flag. See the docs for more details. |
email_type | optional GenerateEmailType | Optionally specify the type of email to generate. The types specified determine the contents on the left side of the @. |
invalid_email_action | optional InvalidEmailAction | Optionally specify an action in the result of an invalid email. An invalid email is one that does not conform to RFC 5322. |
TransformFirstName
Name | Type | Description |
---|---|---|
preserve_length | optional bool |
TransformFloat64
Name | Type | Description |
---|---|---|
randomization_range_min | optional double | |
randomization_range_max | optional double |
TransformFullName
Name | Type | Description |
---|---|---|
preserve_length | optional bool |
TransformInt64
Name | Type | Description |
---|---|---|
randomization_range_min | optional int64 | |
randomization_range_max | optional int64 |
TransformInt64PhoneNumber
Name | Type | Description |
---|---|---|
preserve_length | optional bool |
TransformJavascript
Name | Type | Description |
---|---|---|
code | string |
TransformLastName
Name | Type | Description |
---|---|---|
preserve_length | optional bool |
TransformPhoneNumber
Name | Type | Description |
---|---|---|
preserve_length | optional bool |
TransformPiiText
NeosyncCloud/Enterprise only transformer for anonymizing PII Text
Name | Type | Description |
---|---|---|
score_threshold | float | Minimal detection score for determining PII. 0.0-1.0 |
default_anonymizer | PiiAnonymizer | The default anonmyization config used for all instances of detected PII. |
deny_recognizers | repeated PiiDenyRecognizer | Configure deny lists where each word is treated as PII. |
allowed_entities | repeated string | Configure a list of entities to be used for PII analysis. If not provided or empty, all entities are considiered If this is specified, any ad-hoc, or deny_recognizers entity names must also be provided. To see available builtin entities, call the GetPiiTextEntities() RPC method to see what is available for your account. |
allowed_phrases | repeated string | Exact match of PII phrases identified by the analysis engine. |
TransformString
Name | Type | Description |
---|---|---|
preserve_length | optional bool |
TransformerConfig
Name | Type | Description |
---|---|---|
generate_email_config | GenerateEmail | |
transform_email_config | TransformEmail | |
generate_bool_config | GenerateBool | |
generate_card_number_config | GenerateCardNumber | |
generate_city_config | GenerateCity | |
generate_e164_phone_number_config | GenerateE164PhoneNumber | |
generate_first_name_config | GenerateFirstName | |
generate_float64_config | GenerateFloat64 | |
generate_full_address_config | GenerateFullAddress | |
generate_full_name_config | GenerateFullName | |
generate_gender_config | GenerateGender | |
generate_int64_phone_number_config | GenerateInt64PhoneNumber | |
generate_int64_config | GenerateInt64 | |
generate_last_name_config | GenerateLastName | |
generate_sha256hash_config | GenerateSha256Hash | |
generate_ssn_config | GenerateSSN | |
generate_state_config | GenerateState | |
generate_street_address_config | GenerateStreetAddress | |
generate_string_phone_number_config | GenerateStringPhoneNumber | |
generate_string_config | GenerateString | |
generate_unixtimestamp_config | GenerateUnixTimestamp | |
generate_username_config | GenerateUsername | |
generate_utctimestamp_config | GenerateUtcTimestamp | |
generate_uuid_config | GenerateUuid | |
generate_zipcode_config | GenerateZipcode | |
transform_e164_phone_number_config | TransformE164PhoneNumber | |
transform_first_name_config | TransformFirstName | |
transform_float64_config | TransformFloat64 | |
transform_full_name_config | TransformFullName | |
transform_int64_phone_number_config | TransformInt64PhoneNumber | |
transform_int64_config | TransformInt64 | |
transform_last_name_config | TransformLastName | |
transform_phone_number_config | TransformPhoneNumber | |
transform_string_config | TransformString | |
passthrough_config | Passthrough | |
nullconfig | Null | |
user_defined_transformer_config | UserDefinedTransformerConfig | |
generate_default_config | GenerateDefault | |
transform_javascript_config | TransformJavascript | |
generate_categorical_config | GenerateCategorical | |
transform_character_scramble_config | TransformCharacterScramble | |
generate_javascript_config | GenerateJavascript | |
generate_country_config | GenerateCountry | |
transform_pii_text_config | TransformPiiText | NeosyncCloud/Enterprise only transformer for anonymizing PII Text |
generate_business_name_config | GenerateBusinessName | |
generate_ip_address_config | GenerateIpAddress |
UpdateUserDefinedTransformerRequest
Name | Type | Description |
---|---|---|
transformer_id | string | |
name | string | |
description | string | |
transformer_config | TransformerConfig |
UpdateUserDefinedTransformerResponse
Name | Type | Description |
---|---|---|
transformer | UserDefinedTransformer |
UserDefinedTransformer
Name | Type | Description |
---|---|---|
id | string | |
name | string | |
description | string | |
data_type | TransformerDataType | This property is readonly and is calculated based off the origin system transformer |
source | TransformerSource | |
config | TransformerConfig | |
created_at | google.protobuf.Timestamp | |
updated_at | google.protobuf.Timestamp | |
account_id | string | |
data_types | repeated TransformerDataType |
UserDefinedTransformerConfig
Name | Type | Description |
---|---|---|
id | string |
ValidateUserJavascriptCodeRequest
Name | Type | Description |
---|---|---|
account_id | string | @deprecated - an account isn't required to validate js code |
code | string |
ValidateUserJavascriptCodeResponse
Name | Type | Description |
---|---|---|
valid | bool |
ValidateUserRegexCodeRequest
Name | Type | Description |
---|---|---|
account_id | string | @deprecated - an account isn't required to validate js code |
user_provided_regex | string |
ValidateUserRegexCodeResponse
Name | Type | Description |
---|---|---|
valid | bool |
Enums
GenerateEmailType
Name | Number | Description |
---|---|---|
GENERATE_EMAIL_TYPE_UNSPECIFIED | 0 | Unspecified defaults to uuid v4 |
GENERATE_EMAIL_TYPE_UUID_V4 | 1 | Generates a random uuid v4 (without hyphens) on the left side of the @. This has all the uniqueness guarantees of the standard uuid v4, along with randomized domains. |
GENERATE_EMAIL_TYPE_FULLNAME | 2 | Generates a full name (firstnamelastname) on the left side of the @. This has high cardinality but is not guaranteed to be unique. |
GenerateIpAddressType
Name | Number | Description |
---|---|---|
GENERATE_IP_ADDRESS_TYPE_UNSPECIFIED | 0 | Unspecified defaults to ipv4 |
GENERATE_IP_ADDRESS_TYPE_V4_PUBLIC | 1 | Generates a random ipv4_public |
GENERATE_IP_ADDRESS_TYPE_V4_PRIVATE_A | 2 | Generates a random ipv4_private_a |
GENERATE_IP_ADDRESS_TYPE_V4_PRIVATE_B | 3 | Generates a random ipv4_private_b |
GENERATE_IP_ADDRESS_TYPE_V4_PRIVATE_C | 4 | Generates a random ipv4_private_c |
GENERATE_IP_ADDRESS_TYPE_V4_LINK_LOCAL | 5 | Generates a random ipv4_linklocal |
GENERATE_IP_ADDRESS_TYPE_V4_MULTICAST | 6 | Generates a random ipv4_multicast |
GENERATE_IP_ADDRESS_TYPE_V4_LOOPBACK | 7 | Generates a random ipv4_loopback |
GENERATE_IP_ADDRESS_TYPE_V6 | 8 | Generates a random ipv6 |
InvalidEmailAction
Action enumeration that dictates what the transform_email transformer should do in the case that it encounters an email that does not conform to RFC 5322
Name | Number | Description |
---|---|---|
INVALID_EMAIL_ACTION_UNSPECIFIED | 0 | Unspecified defaults to reject. |
INVALID_EMAIL_ACTION_REJECT | 1 | If an invalid email is encountered, the transformer will reject the transformation and exit. |
INVALID_EMAIL_ACTION_NULL | 2 | The result of the email will be null. This should only be specified if the destination column is also nullable. |
INVALID_EMAIL_ACTION_PASSTHROUGH | 3 | The result of the email will be identical to the original value, unmodified. |
INVALID_EMAIL_ACTION_GENERATE | 4 | The result of the email will be a brand new generated email. This is the same result as the generate_email transformer. |
PiiAnonymizer.Hash.HashType
Name | Number | Description |
---|---|---|
HASH_TYPE_UNSPECIFIED | 0 | If unspecified, defaults to md5 |
HASH_TYPE_MD5 | 1 | The MD5 Hash Algorithm |
HASH_TYPE_SHA256 | 2 | The SHA256 Hash Algorithm |
HASH_TYPE_SHA512 | 3 | The SHA512 Hash Algorithm |
SupportedJobType
Name | Number | Description |
---|---|---|
SUPPORTED_JOB_TYPE_UNSPECIFIED | 0 | |
SUPPORTED_JOB_TYPE_SYNC | 1 | |
SUPPORTED_JOB_TYPE_GENERATE | 2 |
TransformerDataType
Name | Number | Description |
---|---|---|
TRANSFORMER_DATA_TYPE_UNSPECIFIED | 0 | |
TRANSFORMER_DATA_TYPE_STRING | 1 | |
TRANSFORMER_DATA_TYPE_INT64 | 2 | |
TRANSFORMER_DATA_TYPE_BOOLEAN | 3 | |
TRANSFORMER_DATA_TYPE_FLOAT64 | 4 | |
TRANSFORMER_DATA_TYPE_NULL | 5 | |
TRANSFORMER_DATA_TYPE_ANY | 6 | |
TRANSFORMER_DATA_TYPE_TIME | 7 | |
TRANSFORMER_DATA_TYPE_UUID | 8 |
TransformerSource
Name | Number | Description |
---|---|---|
TRANSFORMER_SOURCE_UNSPECIFIED | 0 | |
TRANSFORMER_SOURCE_PASSTHROUGH | 1 | |
TRANSFORMER_SOURCE_GENERATE_DEFAULT | 2 | |
TRANSFORMER_SOURCE_TRANSFORM_JAVASCRIPT | 3 | |
TRANSFORMER_SOURCE_GENERATE_EMAIL | 4 | |
TRANSFORMER_SOURCE_TRANSFORM_EMAIL | 5 | |
TRANSFORMER_SOURCE_GENERATE_BOOL | 6 | |
TRANSFORMER_SOURCE_GENERATE_CARD_NUMBER | 7 | |
TRANSFORMER_SOURCE_GENERATE_CITY | 8 | |
TRANSFORMER_SOURCE_GENERATE_E164_PHONE_NUMBER | 9 | |
TRANSFORMER_SOURCE_GENERATE_FIRST_NAME | 10 | |
TRANSFORMER_SOURCE_GENERATE_FLOAT64 | 11 | |
TRANSFORMER_SOURCE_GENERATE_FULL_ADDRESS | 12 | |
TRANSFORMER_SOURCE_GENERATE_FULL_NAME | 13 | |
TRANSFORMER_SOURCE_GENERATE_GENDER | 14 | |
TRANSFORMER_SOURCE_GENERATE_INT64_PHONE_NUMBER | 15 | |
TRANSFORMER_SOURCE_GENERATE_INT64 | 16 | |
TRANSFORMER_SOURCE_GENERATE_RANDOM_INT64 | 17 | |
TRANSFORMER_SOURCE_GENERATE_LAST_NAME | 18 | |
TRANSFORMER_SOURCE_GENERATE_SHA256HASH | 19 | |
TRANSFORMER_SOURCE_GENERATE_SSN | 20 | |
TRANSFORMER_SOURCE_GENERATE_STATE | 21 | |
TRANSFORMER_SOURCE_GENERATE_STREET_ADDRESS | 22 | |
TRANSFORMER_SOURCE_GENERATE_STRING_PHONE_NUMBER | 23 | |
TRANSFORMER_SOURCE_GENERATE_STRING | 24 | |
TRANSFORMER_SOURCE_GENERATE_RANDOM_STRING | 25 | |
TRANSFORMER_SOURCE_GENERATE_UNIXTIMESTAMP | 26 | |
TRANSFORMER_SOURCE_GENERATE_USERNAME | 27 | |
TRANSFORMER_SOURCE_GENERATE_UTCTIMESTAMP | 28 | |
TRANSFORMER_SOURCE_GENERATE_UUID | 29 | |
TRANSFORMER_SOURCE_GENERATE_ZIPCODE | 30 | |
TRANSFORMER_SOURCE_TRANSFORM_E164_PHONE_NUMBER | 31 | |
TRANSFORMER_SOURCE_TRANSFORM_FIRST_NAME | 32 | |
TRANSFORMER_SOURCE_TRANSFORM_FLOAT64 | 33 | |
TRANSFORMER_SOURCE_TRANSFORM_FULL_NAME | 34 | |
TRANSFORMER_SOURCE_TRANSFORM_INT64_PHONE_NUMBER | 35 | |
TRANSFORMER_SOURCE_TRANSFORM_INT64 | 36 | |
TRANSFORMER_SOURCE_TRANSFORM_LAST_NAME | 37 | |
TRANSFORMER_SOURCE_TRANSFORM_PHONE_NUMBER | 38 | |
TRANSFORMER_SOURCE_TRANSFORM_STRING | 39 | |
TRANSFORMER_SOURCE_GENERATE_NULL | 40 | |
TRANSFORMER_SOURCE_GENERATE_CATEGORICAL | 42 | |
TRANSFORMER_SOURCE_TRANSFORM_CHARACTER_SCRAMBLE | 43 | |
TRANSFORMER_SOURCE_USER_DEFINED | 44 | |
TRANSFORMER_SOURCE_GENERATE_JAVASCRIPT | 45 | |
TRANSFORMER_SOURCE_GENERATE_COUNTRY | 46 | |
TRANSFORMER_SOURCE_TRANSFORM_PII_TEXT | 47 | NeosyncCloud/Enterprise only transformer for anonymizing PII Text |
TRANSFORMER_SOURCE_GENERATE_BUSINESS_NAME | 48 | |
TRANSFORMER_SOURCE_GENERATE_IP_ADDRESS | 49 |
Services
TransformersService
GetSystemTransformers
Method | GetSystemTransformers |
---|---|
Request | GetSystemTransformersRequest |
Response | GetSystemTransformersResponse |
Description |
GetSystemTransformerBySource
Method | GetSystemTransformerBySource |
---|---|
Request | GetSystemTransformerBySourceRequest |
Response | GetSystemTransformerBySourceResponse |
Description |
GetUserDefinedTransformers
Method | GetUserDefinedTransformers |
---|---|
Request | GetUserDefinedTransformersRequest |
Response | GetUserDefinedTransformersResponse |
Description |
GetUserDefinedTransformerById
Method | GetUserDefinedTransformerById |
---|---|
Request | GetUserDefinedTransformerByIdRequest |
Response | GetUserDefinedTransformerByIdResponse |
Description |
CreateUserDefinedTransformer
Method | CreateUserDefinedTransformer |
---|---|
Request | CreateUserDefinedTransformerRequest |
Response | CreateUserDefinedTransformerResponse |
Description |
DeleteUserDefinedTransformer
Method | DeleteUserDefinedTransformer |
---|---|
Request | DeleteUserDefinedTransformerRequest |
Response | DeleteUserDefinedTransformerResponse |
Description |
UpdateUserDefinedTransformer
Method | UpdateUserDefinedTransformer |
---|---|
Request | UpdateUserDefinedTransformerRequest |
Response | UpdateUserDefinedTransformerResponse |
Description |
IsTransformerNameAvailable
Method | IsTransformerNameAvailable |
---|---|
Request | IsTransformerNameAvailableRequest |
Response | IsTransformerNameAvailableResponse |
Description |
ValidateUserJavascriptCode
Method | ValidateUserJavascriptCode |
---|---|
Request | ValidateUserJavascriptCodeRequest |
Response | ValidateUserJavascriptCodeResponse |
Description |
ValidateUserRegexCode
Method | ValidateUserRegexCode |
---|---|
Request | ValidateUserRegexCodeRequest |
Response | ValidateUserRegexCodeResponse |
Description |
GetTransformPiiEntities
Method | GetTransformPiiEntities |
---|---|
Request | GetTransformPiiEntitiesRequest |
Response | GetTransformPiiEntitiesResponse |
Description | Retrieve a list of available Pii entities for use with the TransformPiiText transformer |