Problem with DB encryption on Release (1.5.4)

[madtocc]

Hi,
I was using the sembast with the xxtea codec and it was working great for debug/release on the latest stable version of flutter (1.2.1). But after upgrading to the latest stable version (1.5.4) the release version doesn't work it throws [2] Invalid codec signature when I open the DB. Do you have any idea why this is happening?
Thank you!

[alextekartik]

Arg. Indeed that's not good. And impossible to say what the cause is here as it may simply be a password issue. If you have a way to send me the data and the password or if you could debug it yourself, that could be a great first investigation test. I'll try a repro on my side as well.

[alextekartik]

After some tests, I found that indeed xxtea gives different results on debug and release!

xxtea.encryptToString('value', 'test')

gives 'NMvXe7aiJfgaFlsx' on debug and something different (randomly) on release (but that could be possible). But it also cannot decrypt the debug value:

xxtea.decryptToString('NMvXe7aiJfgaFlsx', 'test')

gives something different each time ('Falue', '\nalue'...) where the first letter is messed up. I don't use xxtea personnally. I have used it as a sample here. I filled an issue here xxtea/xxtea-dart#2

[madtocc]

Thank you very much for your feedback and prompt reply... I was going to dig deeper in the weekend, but you have found the issue already. Well let's wait for his reply :)

[madtocc]

well, almost 2 weeks and the developer of xxtea didn't reply... so I'd suggest to use another source of encryption, maybe this?
I've tried and it works. You can easily add the codec or any user could do it, it's pretty straightforward. If anyone needs to encrypt for a release, you can use this helper temporary and open the database like

await dbFactory.openDatabase(dbPath,codec: getEncryptSembastCodec(password: pass););

PS: the default encryption is Salsa20 (you will need a key of 16 chars). You can use AES instead of Salsa20 but keep mind you'll need a key of 32 chars.

import 'dart:convert';
import 'package:meta/meta.dart';
import 'package:sembast/sembast.dart';
import 'package:encrypt/encrypt.dart';

class _EncryptEncoder extends Converter<Map<String, dynamic>, String> {
  final String key;
  final String signature;
  _EncryptEncoder(this.key, this.signature);

  @override
  String convert(Map<String, dynamic> input) {
    String encoded;
    switch (signature) {
      case "Salsa20":
        encoded = Encrypter(Salsa20(Key.fromUtf8(key)))
            .encrypt(json.encode(input), iv: IV.fromLength(8))
            .base64;
        break;
      case "AES":
        encoded = Encrypter(AES(Key.fromUtf8(key)))
            .encrypt(json.encode(input), iv: IV.fromLength(16))
            .base64;
        break;
      default:
        throw FormatException('invalid $signature');
        break;
    }
    return encoded;
  }
}

class _EncryptDecoder extends Converter<String, Map<String, dynamic>> {
  final String key;
  final String signature;
  _EncryptDecoder(this.key, this.signature);

  @override
  Map<String, dynamic> convert(String input) {
    var decoded;
    switch (signature) {
      case "Salsa20":
        decoded = json.decode(Encrypter(Salsa20(Key.fromUtf8(key)))
            .decrypt64(input, iv: IV.fromLength(8)));
        break;
      case "AES":
        decoded = json.decode(Encrypter(AES(Key.fromUtf8(key)))
            .decrypt64(input, iv: IV.fromLength(16)));
        break;
      default:
        break;
    }
    if (decoded is Map) {
      return decoded.cast<String, dynamic>();
    }
    throw FormatException('invalid input $input');
  }
}

class _EncryptCodec extends Codec<Map<String, dynamic>, String> {
  final String signature;
  _EncryptEncoder _encoder;
  _EncryptDecoder _decoder;
  _EncryptCodec(String password, this.signature) {
    _encoder = _EncryptEncoder(password, signature);
    _decoder = _EncryptDecoder(password, signature);
  }

  @override
  Converter<String, Map<String, dynamic>> get decoder => _decoder;

  @override
  Converter<Map<String, dynamic>, String> get encoder => _encoder;
}

// Salsa20 (16 length key required) or AES (32 length key required)
SembastCodec getEncryptSembastCodec(
        {@required String password, String signature = "Salsa20"}) =>
    SembastCodec(
        signature: signature, codec: _EncryptCodec(password, signature));

[alextekartik]

Yeap indeed. I agree that having an example that does not work on release is not good. I will look into it. The idea is to propose a solution that can be used just by copying one file (+ dependencies). I'll try your proposed encoder. I'd like to have an example that does not require a specific password length. Is there a way to generate a password from any string (hash, padding...)?

[madtocc]

yest it does, but you'll need the password hashing lib from the same guy. I didn't test it but it should do the job.

[alextekartik]

Thanks again for your example as I was able to use it almost as is in my tests (besides the password length issue). I remove some features (Salsa20 only) and used sha256 to generate the password - that seems to work, right?) and kept it as small as possible to act as an example.
https://github.com/tekartik/sembast.dart/blob/master/sembast/test/encrypt_codec.dart

I removed all the references to xxtea.

[madtocc]

sure, glad to help! I have tested and it's working. You can close this issue since it's solved :)
I just have a small suggestion (I don't know if it has a a noticeable impact, though): maybe change the sha256 to md5 or sha-1 which are computed faster. This salsa20 truncates the password string after the 16th char anyway...

[GreenAppers]

The IV.fromLength(8) in encrypt_codec.dart is using 0 for the IV every time. It should really be using a random IV, and prepending the IV to the cipher text. Something like this:

import 'package:pointycastle/api.dart';
import 'package:pointycastle/stream/salsa20.dart';

class Salsa20Encoder extends Converter<Map<String, dynamic>, String> {
  Uint8List password;
  Salsa20Encoder(this.password) {
    assert(password.length == 32);
  }

  @override
  String convert(Map<String, dynamic> input) {
    Uint8List initialValue = randBytes(8);
    String encoded = base64.encode(initialValue);
    assert(encoded.length == 12);

    final Salsa20Engine salsa20 = Salsa20Engine();
    salsa20.init(true,
        ParametersWithIV<KeyParameter>(KeyParameter(password), initialValue));
    encoded += base64.encode(salsa20.process(utf8.encode(jsonEncode(input))));
    return encoded;
  }
} 

class Salsa20Decoder extends Converter<String, Map<String, dynamic>> {
  Uint8List password;
  Salsa20Decoder(this.password) {
    assert(password.length == 32);
  }

  @override
  Map<String, dynamic> convert(String input) {
    assert(input.length >= 12);
    Uint8List initialValue = base64.decode(input.substring(0, 12));
    input = input.substring(12);

    final Salsa20Engine salsa20 = Salsa20Engine();
    salsa20.init(false,
        ParametersWithIV<KeyParameter>(KeyParameter(password), initialValue));
    var decoded =
        json.decode(utf8.decode(salsa20.process(base64.decode(input))));
    if (decoded is Map) {
      return decoded.cast<String, dynamic>();
    }
    throw FormatException('invalid input $input');
  }
}

[alextekartik]

Thanks for the suggestion @GreenAppers I will check how to integrate that. (PR are welcome!)

[alextekartik]

Thanks again @GreenAppers and @madtocc I think I was able to improve the encrypt codec example. To note that I have not done any benchmark nor extensive extensing, especially on big database so it will remain an example, not a reference (i.e. its implementation and format might change). I kept the solution in one file (+ dependencies) so that it can be easily tried and tuned.

I have also used md5 instead of sha256 since it generates a 16 bytes blobs that matches what Salsa20 expect as a password (and we don't even store the password)