Skip to content

Renaming Format to Encoding for parquet encodings #207

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
avalerio-tkd merged 3 commits into from
Jan 24, 2026
Merged

Renaming Format to Encoding for parquet encodings #207

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
bbe9cdd
- Straight rename of Format to Encoding to match parquet::Encoding
avalerio-tkd Jan 23, 2026
e0e5a1f
- Renamed ConvertAndEncrypt and ConvertAndDecrypt to DecodeAndEncrypt…
avalerio-tkd Jan 24, 2026
551bd81
- Removing TODO note now that methods were renamed.
avalerio-tkd Jan 24, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 4 additions & 4 deletions src/client/dbps_api_client.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -183,7 +183,7 @@ EncryptApiResponse DBPSApiClient::Encrypt(
Type::type datatype,
const std::optional<int>& datatype_length,
CompressionCodec::type compression,
Format::type format,
Encoding::type encoding,
const std::map<std::string, std::string>& encoding_attributes,
CompressionCodec::type encrypted_compression,
const std::string& key_id,
Expand All @@ -195,7 +195,7 @@ EncryptApiResponse DBPSApiClient::Encrypt(
json_request.datatype_ = datatype;
json_request.datatype_length_ = datatype_length;
json_request.compression_ = compression;
json_request.format_ = format;
json_request.encoding_ = encoding;
json_request.encoding_attributes_ = encoding_attributes;
json_request.encrypted_compression_ = encrypted_compression;
json_request.key_id_ = key_id;
Expand Down Expand Up @@ -265,7 +265,7 @@ DecryptApiResponse DBPSApiClient::Decrypt(
Type::type datatype,
const std::optional<int>& datatype_length,
CompressionCodec::type compression,
Format::type format,
Encoding::type encoding,
const std::map<std::string, std::string>& encoding_attributes,
CompressionCodec::type encrypted_compression,
const std::string& key_id,
Expand All @@ -278,7 +278,7 @@ DecryptApiResponse DBPSApiClient::Decrypt(
json_request.datatype_ = datatype;
json_request.datatype_length_ = datatype_length;
json_request.compression_ = compression;
json_request.format_ = format;
json_request.encoding_ = encoding;
json_request.encoding_attributes_ = encoding_attributes;
json_request.encrypted_compression_ = encrypted_compression;
json_request.key_id_ = key_id;
Expand Down
12 changes: 6 additions & 6 deletions src/client/dbps_api_client.h
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -164,7 +164,7 @@ class DBPSApiClient {
* @param column_name Name of the database column for which this data is being encrypted
* @param datatype The data type of the plaintext (e.g., BYTE_ARRAY, STRING, etc.)
* @param compression Compression algorithm used to compress the plaintext before this call (format of the input)
* @param format Data format specification (Parquet formats supported)
* @param encoding Data encoding specification (Parquet encodings supported)
* @param encoding_attributes Map of string key-value pairs containing encoding-specific attributes
* @param encrypted_compression Compression algorithm to be used to compress the encrypted data (format of the output)
* @param key_id Identifier for the encryption key to be used (not the key itself)
Expand All @@ -173,15 +173,15 @@ class DBPSApiClient {
*
* @return The encryption API response object containing comprehensive information about the call
*
* @note Data types, compression types, and formats as defined in the enums.h file are supported
* @note Data types, compression types, and encodings as defined in the enums.h file are supported
*/
EncryptApiResponse Encrypt(
span<const uint8_t> plaintext,
const std::string& column_name,
Type::type datatype,
const std::optional<int>& datatype_length,
CompressionCodec::type compression,
Format::type format,
Encoding::type encoding,
const std::map<std::string, std::string>& encoding_attributes,
CompressionCodec::type encrypted_compression,
const std::string& key_id,
Expand All @@ -196,7 +196,7 @@ class DBPSApiClient {
* @param column_name Name of the database column for which this data is being decrypted
* @param datatype The data type of the original plaintext (e.g., BYTE_ARRAY, STRING, etc.)
* @param compression Compression algorithm used to compress the encrypted data before this call (format of the input)
* @param format Data format specification (Parquet formats supported)
* @param encoding Data encoding specification (Parquet encodings supported)
* @param encoding_attributes Map of string key-value pairs containing encoding-specific attributes
* @param encrypted_compression Compression algorithm to be used to compress the decrypted data (format of the output)
* @param key_id Identifier for the encryption key to be used for decryption (not the key itself)
Expand All @@ -205,15 +205,15 @@ class DBPSApiClient {
*
* @return The decryption API response object containing comprehensive information about the call
*
* @note Data types, compression types, and formats as defined in the enums.h file are supported
* @note Data types, compression types, and encodings as defined in the enums.h file are supported
*/
DecryptApiResponse Decrypt(
span<const uint8_t> ciphertext,
const std::string& column_name,
Type::type datatype,
const std::optional<int>& datatype_length,
CompressionCodec::type compression,
Format::type format,
Encoding::type encoding,
const std::map<std::string, std::string>& encoding_attributes,
CompressionCodec::type encrypted_compression,
const std::string& key_id,
Expand Down
28 changes: 14 additions & 14 deletions src/client/dbps_api_client_test.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -236,7 +236,7 @@ TEST(DBPSApiClient, DecryptApiResponseGetResponsePlaintextWithValidData) {
json_response.decrypted_value_ = StringToBytes("test@example.com");
json_response.datatype_ = Type::BYTE_ARRAY;
json_response.compression_ = CompressionCodec::UNCOMPRESSED;
json_response.format_ = Format::PLAIN;
json_response.encoding_ = Encoding::PLAIN;
json_response.user_id_ = "test_user";
json_response.role_ = "test_role";
json_response.access_control_ = "test_access";
Expand Down Expand Up @@ -280,7 +280,7 @@ TEST(DBPSApiClient, EncryptWithValidData) {
"value": "dGVzdEBleGFtcGxlLmNvbQ==",
"value_format": {
"compression": "UNCOMPRESSED",
"format": "PLAIN"
"encoding": "PLAIN"
}
},
"data_batch_encrypted": {
Expand Down Expand Up @@ -325,7 +325,7 @@ TEST(DBPSApiClient, EncryptWithValidData) {
Type::BYTE_ARRAY, // datatype
std::nullopt, // datatype_length
CompressionCodec::UNCOMPRESSED, // compression
Format::PLAIN, // format
Encoding::PLAIN, // encoding
std::map<std::string, std::string>{}, // encoding_attributes
CompressionCodec::UNCOMPRESSED, // encrypted_compression
"test_key_123", // key_id
Expand Down Expand Up @@ -366,7 +366,7 @@ TEST(DBPSApiClient, DecryptWithValidData) {
},
"value_format": {
"compression": "UNCOMPRESSED",
"format": "PLAIN"
"encoding": "PLAIN"
}
},
"data_batch_encrypted": {
Expand All @@ -388,7 +388,7 @@ TEST(DBPSApiClient, DecryptWithValidData) {
"value": "dGVzdEBleGFtcGxlLmNvbQ==",
"value_format": {
"compression": "UNCOMPRESSED",
"format": "PLAIN"
"encoding": "PLAIN"
}
},
"access": {
Expand Down Expand Up @@ -417,7 +417,7 @@ TEST(DBPSApiClient, DecryptWithValidData) {
Type::BYTE_ARRAY, // datatype
std::nullopt, // datatype_length
CompressionCodec::UNCOMPRESSED, // compression
Format::PLAIN, // format
Encoding::PLAIN, // encoding
std::map<std::string, std::string>{}, // encoding_attributes
CompressionCodec::UNCOMPRESSED, // encrypted_compression
"test_key_123", // key_id
Expand Down Expand Up @@ -464,7 +464,7 @@ TEST(DBPSApiClient, EncryptWithInvalidData) {
Type::BYTE_ARRAY, // datatype
std::nullopt, // datatype_length
CompressionCodec::UNCOMPRESSED, // compression
Format::PLAIN, // format
Encoding::PLAIN, // encoding
std::map<std::string, std::string>{}, // encoding_attributes
CompressionCodec::UNCOMPRESSED, // encrypted_compression
"test_key_123", // key_id
Expand Down Expand Up @@ -494,7 +494,7 @@ TEST(DBPSApiClient, DecryptWithInvalidData) {
Type::BYTE_ARRAY, // datatype
std::nullopt, // datatype_length
CompressionCodec::UNCOMPRESSED, // compression
Format::PLAIN, // format
Encoding::PLAIN, // encoding
std::map<std::string, std::string>{}, // encoding_attributes
CompressionCodec::UNCOMPRESSED, // encrypted_compression
"test_key_123", // key_id
Expand All @@ -521,7 +521,7 @@ TEST(DBPSApiClient, EncryptWithInvalidJsonResponse) {
"value": "dGVzdEBleGFtcGxlLmNvbQ==",
"value_format": {
"compression": "UNCOMPRESSED",
"format": "PLAIN"
"encoding": "PLAIN"
}
},
"data_batch_encrypted": {
Expand Down Expand Up @@ -560,7 +560,7 @@ TEST(DBPSApiClient, EncryptWithInvalidJsonResponse) {
Type::BYTE_ARRAY, // datatype
std::nullopt, // datatype_length
CompressionCodec::UNCOMPRESSED, // compression
Format::PLAIN, // format
Encoding::PLAIN, // encoding
std::map<std::string, std::string>{}, // encoding_attributes
CompressionCodec::UNCOMPRESSED, // encrypted_compression
"test_key_123", // key_id
Expand All @@ -585,7 +585,7 @@ TEST(DBPSApiClient, DecryptWithInvalidJsonResponse) {
},
"value_format": {
"compression": "UNCOMPRESSED",
"format": "PLAIN"
"encoding": "PLAIN"
}
},
"data_batch_encrypted": {
Expand Down Expand Up @@ -626,7 +626,7 @@ TEST(DBPSApiClient, DecryptWithInvalidJsonResponse) {
Type::BYTE_ARRAY, // datatype
std::nullopt, // datatype_length
CompressionCodec::UNCOMPRESSED, // compression
Format::PLAIN, // format
Encoding::PLAIN, // encoding
std::map<std::string, std::string>{}, // encoding_attributes
CompressionCodec::UNCOMPRESSED, // encrypted_compression
"test_key_123", // key_id
Expand All @@ -653,7 +653,7 @@ TEST(DBPSApiClient, EncryptWithEncodingAttributes) {
"value": "dGVzdEBleGFtcGxlLmNvbQ==",
"value_format": {
"compression": "UNCOMPRESSED",
"format": "PLAIN",
"encoding": "PLAIN",
"encoding_attributes": {
"page_type": "DATA_PAGE",
"page_encoding": "PLAIN",
Expand Down Expand Up @@ -709,7 +709,7 @@ TEST(DBPSApiClient, EncryptWithEncodingAttributes) {
Type::BYTE_ARRAY, // datatype
std::nullopt, // datatype_length
CompressionCodec::UNCOMPRESSED, // compression
Format::PLAIN, // format
Encoding::PLAIN, // encoding
encoding_attributes, // encoding_attributes
CompressionCodec::UNCOMPRESSED, // encrypted_compression
"test_key_123", // key_id
Expand Down
2 changes: 1 addition & 1 deletion src/common/dbpa_interface.h
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -123,7 +123,7 @@ class DBPS_EXPORT DataBatchProtectionAgentInterface {
* Encrypts the provided plaintext data using the configured encryption parameters.
*
* @param plaintext Binary data to be encrypted, provided as a span of bytes
* @param encoding_attributes A map of string key-values. The plaintext is encoded with a type defined in enums.h Format::type.
* @param encoding_attributes A map of string key-values. The plaintext is encoded with a type defined in enums.h Encoding::type.
* Each encoding type requires additional attributes to be properly decoded. These attributes are specified in the map so an
* implementation can properly interpret and process the input text.
*
Expand Down
2 changes: 1 addition & 1 deletion src/common/dbpa_interface_test.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -108,7 +108,7 @@ TEST(DBPAInterface, AgentEncryptDecrypt) {
MockAgent agent;
std::vector<uint8_t> original = {10, 20, 30};

std::map<std::string, std::string> encoding_attributes = {{"format", "PLAIN"}};
std::map<std::string, std::string> encoding_attributes = {{"page_encoding", "PLAIN"}};
auto encrypted = agent.Encrypt(span<const uint8_t>(original.data(), original.size()), encoding_attributes);
ASSERT_TRUE(encrypted->success());

Expand Down
16 changes: 8 additions & 8 deletions src/common/dbpa_local.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -179,9 +179,9 @@ std::unique_ptr<EncryptionResult> LocalDataBatchProtectionAgent::Encrypt(
return std::make_unique<LocalEncryptionResult>("initialization", *initialized_);
}

// Extract page_encoding from encoding_attributes and convert to Format::type
auto format_opt = dbps::external::ExtractPageEncoding(encoding_attributes);
if (!format_opt.has_value()) {
// Extract page_encoding from encoding_attributes and convert to Encoding::type
auto encoding_opt = dbps::external::ExtractPageEncoding(encoding_attributes);
if (!encoding_opt.has_value()) {
std::cerr << "ERROR: LocalDataBatchProtectionAgent::Encrypt() - page_encoding not found or invalid in encoding_attributes." << std::endl;
return std::make_unique<LocalEncryptionResult>("parameter_validation", "page_encoding not found or invalid in encoding_attributes");
}
Expand All @@ -192,7 +192,7 @@ std::unique_ptr<EncryptionResult> LocalDataBatchProtectionAgent::Encrypt(
datatype_,
datatype_length_,
compression_type_,
format_opt.value(),
encoding_opt.value(),
encoding_attributes,
compression_type_,
column_key_id_,
Expand Down Expand Up @@ -230,9 +230,9 @@ std::unique_ptr<DecryptionResult> LocalDataBatchProtectionAgent::Decrypt(
return std::make_unique<LocalDecryptionResult>("initialization", *initialized_);
}

// Extract page_encoding from encoding_attributes and convert to Format::type
auto format_opt = dbps::external::ExtractPageEncoding(encoding_attributes);
if (!format_opt.has_value()) {
// Extract page_encoding from encoding_attributes and convert to Encoding::type
auto encoding_opt = dbps::external::ExtractPageEncoding(encoding_attributes);
if (!encoding_opt.has_value()) {
std::cerr << "ERROR: LocalDataBatchProtectionAgent::Decrypt() - page_encoding not found or invalid in encoding_attributes." << std::endl;
return std::make_unique<LocalDecryptionResult>("parameter_validation", "page_encoding not found or invalid in encoding_attributes");
}
Expand All @@ -243,7 +243,7 @@ std::unique_ptr<DecryptionResult> LocalDataBatchProtectionAgent::Decrypt(
datatype_,
datatype_length_,
compression_type_,
format_opt.value(),
encoding_opt.value(),
encoding_attributes,
compression_type_,
column_key_id_,
Expand Down
18 changes: 9 additions & 9 deletions src/common/dbpa_remote.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -258,9 +258,9 @@ std::unique_ptr<EncryptionResult> RemoteDataBatchProtectionAgent::Encrypt(span<c
return std::make_unique<RemoteEncryptionResult>(std::move(empty_response));
}

// Extract page_encoding from encoding_attributes and convert to Format::type
auto format_opt = ExtractPageEncoding(encoding_attributes);
if (!format_opt.has_value()) {
// Extract page_encoding from encoding_attributes and convert to Encoding::type
auto encoding_opt = ExtractPageEncoding(encoding_attributes);
if (!encoding_opt.has_value()) {
std::cerr << "ERROR: RemoteDataBatchProtectionAgent::Encrypt() - page_encoding not found or invalid in encoding_attributes." << std::endl;
auto empty_response = std::make_unique<EncryptApiResponse>();
empty_response->SetApiClientError("page_encoding not found or invalid in encoding_attributes");
Expand All @@ -274,7 +274,7 @@ std::unique_ptr<EncryptionResult> RemoteDataBatchProtectionAgent::Encrypt(span<c
datatype_,
datatype_length_,
compression_type_,
format_opt.value(),
encoding_opt.value(),
encoding_attributes,
compression_type_,
column_key_id_,
Expand Down Expand Up @@ -314,9 +314,9 @@ std::unique_ptr<DecryptionResult> RemoteDataBatchProtectionAgent::Decrypt(span<c
return std::make_unique<RemoteDecryptionResult>(std::move(empty_response));
}

// Extract page_encoding from encoding_attributes and convert to Format::type
auto format_opt = ExtractPageEncoding(encoding_attributes);
if (!format_opt.has_value()) {
// Extract page_encoding from encoding_attributes and convert to Encoding::type
auto encoding_opt = ExtractPageEncoding(encoding_attributes);
if (!encoding_opt.has_value()) {
std::cerr << "ERROR: RemoteDataBatchProtectionAgent::Decrypt() - page_encoding not found or invalid in encoding_attributes." << std::endl;
auto empty_response = std::make_unique<DecryptApiResponse>();
empty_response->SetApiClientError("page_encoding not found or invalid in encoding_attributes");
Expand All @@ -330,7 +330,7 @@ std::unique_ptr<DecryptionResult> RemoteDataBatchProtectionAgent::Decrypt(span<c
datatype_,
datatype_length_,
compression_type_,
format_opt.value(),
encoding_opt.value(),
encoding_attributes,
compression_type_,
column_key_id_,
Expand All @@ -341,7 +341,7 @@ std::unique_ptr<DecryptionResult> RemoteDataBatchProtectionAgent::Decrypt(span<c
);

// Validate that response fields match request fields
// TODO: Add validation for format when these are expanded beyond PLAIN.
// TODO: Add validation for encoding when these are expanded beyond PLAIN.
if (response.Success()) {
const auto& response_attrs = response.GetResponseAttributes();

Expand Down
6 changes: 3 additions & 3 deletions src/common/dbpa_remote_test.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -502,7 +502,7 @@ TEST_F(RemoteDataBatchProtectionAgentTest, SuccessfulDecryption) {
"\"debug\":{\"reference_id\":\"123\"},"
"\"data_batch\":{"
"\"datatype_info\":{\"datatype\":\"BYTE_ARRAY\"},"
"\"value_format\":{\"compression\":\"UNCOMPRESSED\",\"format\":\"PLAIN\"},"
"\"value_format\":{\"compression\":\"UNCOMPRESSED\",\"encoding\":\"PLAIN\"},"
"\"value\":\"dGVzdF9kYXRh\""
"}}",
""
Expand Down Expand Up @@ -551,7 +551,7 @@ TEST_F(RemoteDataBatchProtectionAgentTest, DecryptionFieldMismatch) {
"\"debug\":{\"reference_id\":\"123\"},"
"\"data_batch\":{"
"\"datatype_info\":{\"datatype\":\"INT32\"},"
"\"value_format\":{\"compression\":\"UNCOMPRESSED\",\"format\":\"PLAIN\"},"
"\"value_format\":{\"compression\":\"UNCOMPRESSED\",\"encoding\":\"PLAIN\"},"
"\"value\":\"dGVzdF9kYXRh\""
"}}",
"datatype mismatch",
Expand All @@ -564,7 +564,7 @@ TEST_F(RemoteDataBatchProtectionAgentTest, DecryptionFieldMismatch) {
"\"debug\":{\"reference_id\":\"123\"},"
"\"data_batch\":{"
"\"datatype_info\":{\"datatype\":\"BYTE_ARRAY\"},"
"\"value_format\":{\"compression\":\"GZIP\",\"format\":\"PLAIN\"},"
"\"value_format\":{\"compression\":\"GZIP\",\"encoding\":\"PLAIN\"},"
"\"value\":\"dGVzdF9kYXRh\""
"}}",
"compression mismatch",
Expand Down
Loading