Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update quickstart guide #792

Open
Thrameos opened this issue Jul 16, 2020 · 3 comments
Open

Update quickstart guide #792

Thrameos opened this issue Jul 16, 2020 · 3 comments
Labels
documentation Issue with our documentation (misleading, missing, bad)

Comments

@Thrameos
Copy link
Contributor

It appears that lambda as well as a few of the other late added features have not yet made the quickstart guide. This should be updated.

@Thrameos Thrameos added the documentation Issue with our documentation (misleading, missing, bad) label Jul 16, 2020
@theadroit-m
Copy link

Additionally, i see from the change log that there its faster to pass arrays to-and-from python/java. I did not find any documentation about this. Can we get some code samples for this?

In my usecase, I am trying to pass a byte array from java to python and back (as a byte array). Any pointers would help.

Blurb from documentation:

Improved speed on transfer of lists, tuples, buffers to arrays of Java primitives by a factor of 4 to 100 depending on the data type. The conversion uses optimized path for memory buffers, rather than the Sequence API. When a Python buffer is encountered only the first element is checked for conversion as Python buffers are homogeneous.
Corrected symbol problem with Python 3.5.3. PySlice_Unpack was introduced in a later patch release and should not have been used.

@Thrameos
Copy link
Contributor Author

Most of these behaviors are automatic. Anything with the appropriate byte transfer is able to be transferred in bulk. These include memoryview, bytes, bytearray, and numpy array. If you can show me some sample code with your use case I can give specifics. Python has several APIs for transfer of sequences. For anything which supports the buffer API we can optimize by checking only the first element for conversion and then using the memcpy type API. Unfortunately, Java does not support pinning or write through behaviors so, views of Java memory are read only.

@theadroit-m
Copy link

My current implementation looks as follows:

def to_json(self) -> str:
		# java object’s to json is called
		
        return javaObject.toJson()

@staticmethod
def from_json(json_object: string) -> “javaObject”:
		# The java object is called to load the object from json
        return JavaObject.fromJson(json_object)

As you can expect, the performance of converting an object to #json is high, i implemented a binary serializer on the java side (ProtoStuff). This improved the performance on the java side by 10 fold. I am trying to update the the contract on python class to use the binary serializer in hopes to improve the performance of my program.

The java class has new methods:

public byte[] serialize() {
        // ser
        final byte[] data;
        ...
        data = ProtostuffIOUtil.toByteArray(javaObject, schema, buffer);
        ...
        return data;
}

public static JavaObject deserialize(byte[] data) {
        // deser
        ...
        return new JavaObject(...);
}

Any tips here would be great.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
documentation Issue with our documentation (misleading, missing, bad)
Projects
None yet
Development

No branches or pull requests

2 participants