StreamEntry support Binary

[thachlp]

Issue: #3566

[sazzad16]

We must not break the existing implementations.

[thachlp]

We must not break the existing implementations.

Hi, @sazzad16, thanks for the review 🙇
How do we resolve this issue?

[sazzad16]

How do we resolve this issue?

By creating new methods.

---

I just realized what your title says:

StreamEntry support Binary instead of String

Actually we want StreamEntry binary support while keeping the String ones.

[thachlp]

How do we resolve this issue?

By creating new methods.

I just realized what your title says:

StreamEntry support Binary instead of String

Actually we want StreamEntry binary support while keeping the String ones.

Thanks, I will update.

[thachlp]

Hi @sazzad16, as I understand, there is xadd support Map<byte[], byte[]> hash already.

  public final CommandObject<byte[]> xadd(byte[] key, XAddParams params, Map<byte[], byte[]> hash) {
    return new CommandObject<>(addFlatMapArgs(commandArguments(XADD).key(key).addParams(params), hash),
        BuilderFactory.BINARY);
  }

[sazzad16]

@thachlp StreamEntry has nothing to do with XADD.

[jfloodnet]

Now that this package supports binary value in stream entries, would someone be able to update spark redis to support binary columns also?

It is currently only Map<string, string> https://github.com/RedisLabs/spark-redis/blob/master/src/main/scala/com/redislabs/provider/redis/streaming/RedisStreamReceiver.scala#L108

Which causes the following error:

java.lang.ClassCastException: class org.apache.spark.unsafe.types.UTF8String cannot be cast to class [B (org.apache.spark.unsafe.types.UTF8String is in unnamed module of loader 'app'; [B is in module java.base of loader 'bootstrap') at org.apache.spark.sql.catalyst.expressions.BaseGenericInternalRow.getBinary(rows.scala:46)

If your stream entry schema has binary data:

    public static StructType redisSchema() {
        return new StructType(new StructField[]{
                DataTypes.createStructField("msg", DataTypes.BinaryType, true)
        });
    }

[sazzad16]

@jfloodnet This PR is closed by OP without being merged.

[jfloodnet]

@jfloodnet This PR is closed by OP without being merged.

oh that's true. @thachlp what was the reason you closed this PR?

[thachlp]

@jfloodnet I am just ambiguous about doing it, but If it is important, I will continue on it now, I will ask @sazzad16 for help.