Cython examples?

[luk-f-a]

Hi, thanks a lot for making ROPTLIB! Since I had to use it in python, I created some Cython interfaces for this purpose. It's not a generic Python interface, but it could be useful for others. Would you be interested in adding a Cython examples folder? I could clone the repo and send you a PR.

Regards!

[whuang08]

Thank you very much for the work on python interface.
I believe it is useful. We also thought about creating a python interface. But due to our limited experience on python, we are not able to finish this.
Yes, I'm very interested in adding a such folder. It would be nice if you can write a instruction telling users how to use the python interface. If you don't have objections, I will add you as a contributor (provided that you give me your information) and add the instruction to the next version of the user manual.

Best!

Wen

[luk-f-a]

Hi Wen, I am encouraged by your positive response.

The way I see it is that there at least 3 types of python interfaces that could be created, depending on how much work the user needs to do in C++.

• The more limited interface (which is the one I have already created and will submit an example soon), is a wrapper on user-defined C++ code. This requires the user to setup a C++ development environment and to create a fully fledged C++ implementation of what they are trying to do, and then applying a thin wrapper to be able to use it from Python.
• For users comfortable with Cython and not looking to develop in C++, the next option would be a better fit. It would require writing the "main" function (the equivalent of what you have in the test folder) in cython. This function calls existing ROPTLIB objects from cython. The output is a compiled library that the user can then import in python. I am trying to create an example of this approach.
• A full Python interface would allow calling every object from python directly. I believe this is similar to what you already have for matlab and Julia. Doing this for Python would require either a lot of manual work or some automated tool like SWIG. I have no experience with this type of work, and it is not what I am able to contribute.

[luk-f-a]

I have created two branches, each with an example. Unfortunately neither is working due to so strange errors that I did not encounter before (maybe I was lucky before). I'll look into them later.

[whuang08]

Thank you very much!
If I understand correctly, in the current version of the python interface, users need to write problem-related-objects in C++. When this is done, users can use python interface to solve the optimization problem in python.

It seems that the two examples that you tried to merge are conflicted. I just downloaded them, put them in two folders and committed them to ROPTLIB.

Best,

Wen

[luk-f-a]

yes, I am sorry, I developed them in separate branches but did not realize that they would conflict.
Example 2 should be the more straightforward of two and easy to get running, however there's a problem that I have not been able to solve. I think it's related to the compilation of libropt.so. Maybe you can help me.
I am getting an error that says
ImportError: /.../ROPTLIB/PythonExamples/PySparsePCA.cpython-36m-x86_64-linux-gnu.so: undefined symbol: _Z13testSparsePCAPdS_llldS_S_

I do not know much about c++ compilation, but I was googling this issue and I came across the readelf command. So I ran readelf -Ws libropt.so with the following result

`Symbol table '.dynsym' contains 10 entries:
   Num:    Value          Size Type    Bind   Vis      Ndx Name
     0: 0000000000000000     0 NOTYPE  LOCAL  DEFAULT  UND 
     1: 0000000000000000     0 NOTYPE  WEAK   DEFAULT  UND __cxa_finalize
     2: 0000000000000000     0 NOTYPE  WEAK   DEFAULT  UND _ITM_registerTMCloneTable
     3: 0000000000000000     0 NOTYPE  WEAK   DEFAULT  UND _ITM_deregisterTMCloneTable
     4: 0000000000000000     0 NOTYPE  WEAK   DEFAULT  UND __gmon_start__
     5: 0000000000201020     0 NOTYPE  GLOBAL DEFAULT   17 _edata
     6: 0000000000201028     0 NOTYPE  GLOBAL DEFAULT   18 _end
     7: 0000000000000408     0 FUNC    GLOBAL DEFAULT    6 _init
     8: 0000000000201020     0 NOTYPE  GLOBAL DEFAULT   18 __bss_start
     9: 000000000000051c     0 FUNC    GLOBAL DEFAULT   10 _fini

Symbol table '.symtab' contains 44 entries:
   Num:    Value          Size Type    Bind   Vis      Ndx Name
     0: 0000000000000000     0 NOTYPE  LOCAL  DEFAULT  UND 
     1: 0000000000000190     0 SECTION LOCAL  DEFAULT    1 
     2: 00000000000001b8     0 SECTION LOCAL  DEFAULT    2 
     3: 00000000000001f0     0 SECTION LOCAL  DEFAULT    3 
     4: 00000000000002e0     0 SECTION LOCAL  DEFAULT    4 
     5: 0000000000000360     0 SECTION LOCAL  DEFAULT    5 
     6: 0000000000000408     0 SECTION LOCAL  DEFAULT    6 
     7: 0000000000000420     0 SECTION LOCAL  DEFAULT    7 
     8: 0000000000000430     0 SECTION LOCAL  DEFAULT    8 
     9: 0000000000000440     0 SECTION LOCAL  DEFAULT    9 
    10: 000000000000051c     0 SECTION LOCAL  DEFAULT   10 
    11: 0000000000000528     0 SECTION LOCAL  DEFAULT   11 
    12: 0000000000200e80     0 SECTION LOCAL  DEFAULT   12 
    13: 0000000000200e88     0 SECTION LOCAL  DEFAULT   13 
    14: 0000000000200e90     0 SECTION LOCAL  DEFAULT   14 
    15: 0000000000200fe0     0 SECTION LOCAL  DEFAULT   15 
    16: 0000000000201000     0 SECTION LOCAL  DEFAULT   16 
    17: 0000000000201018     0 SECTION LOCAL  DEFAULT   17 
    18: 0000000000201020     0 SECTION LOCAL  DEFAULT   18 
    19: 0000000000000000     0 SECTION LOCAL  DEFAULT   19 
    20: 0000000000000000     0 FILE    LOCAL  DEFAULT  ABS crtstuff.c
    21: 0000000000000440     0 FUNC    LOCAL  DEFAULT    9 deregister_tm_clones
    22: 0000000000000480     0 FUNC    LOCAL  DEFAULT    9 register_tm_clones
    23: 00000000000004d0     0 FUNC    LOCAL  DEFAULT    9 __do_global_dtors_aux
    24: 0000000000201020     1 OBJECT  LOCAL  DEFAULT   18 completed.7696
    25: 0000000000200e88     0 OBJECT  LOCAL  DEFAULT   13 __do_global_dtors_aux_fini_array_entry
    26: 0000000000000510     0 FUNC    LOCAL  DEFAULT    9 frame_dummy
    27: 0000000000200e80     0 OBJECT  LOCAL  DEFAULT   12 __frame_dummy_init_array_entry
    28: 0000000000000000     0 FILE    LOCAL  DEFAULT  ABS crtstuff.c
    29: 0000000000000528     0 OBJECT  LOCAL  DEFAULT   11 __FRAME_END__
    30: 0000000000000000     0 FILE    LOCAL  DEFAULT  ABS 
    31: 0000000000200e90     0 OBJECT  LOCAL  DEFAULT   14 _DYNAMIC
    32: 0000000000201020     0 OBJECT  LOCAL  DEFAULT   17 __TMC_END__
    33: 0000000000201018     0 OBJECT  LOCAL  DEFAULT   17 __dso_handle
    34: 0000000000201000     0 OBJECT  LOCAL  DEFAULT   16 _GLOBAL_OFFSET_TABLE_
    35: 0000000000000000     0 NOTYPE  WEAK   DEFAULT  UND __cxa_finalize
    36: 0000000000000408     0 FUNC    GLOBAL DEFAULT    6 _init
    37: 0000000000000000     0 NOTYPE  WEAK   DEFAULT  UND _ITM_registerTMCloneTable
    38: 0000000000000000     0 NOTYPE  WEAK   DEFAULT  UND _ITM_deregisterTMCloneTable
    39: 0000000000201020     0 NOTYPE  GLOBAL DEFAULT   18 __bss_start
    40: 000000000000051c     0 FUNC    GLOBAL DEFAULT   10 _fini
    41: 0000000000201020     0 NOTYPE  GLOBAL DEFAULT   17 _edata
    42: 0000000000201028     0 NOTYPE  GLOBAL DEFAULT   18 _end
    43: 0000000000000000     0 NOTYPE  WEAK   DEFAULT  UND __gmon_start__

I think there might be a problem in the compilation of libropt.so (using the provided make file). When I compiled it using a cmake file that CLion created for me, I got more than 600 symbols (I won't paste all of them here). In this file (the cmake version) I can recognize the name of the symbols, among them _GLOBAL__sub_I_SolversLS.cpp, _ZN7ROPTLIB14ProductElement21ObtainWriteEntireDataEv, _ZN7ROPTLIB7StiefelC1Eii. But in the libropt.so generated by the makefile, there are only 40-something symbols and none of them have names that relate to ROPTLIB classes.
Have you been able to use libropt.so yourself? Is it possible that it is missing symbols?

[whuang08]

I did not use libropt.so before. But this option was recommended by another user (Erik Kruus).

By looking into the makefile, compiling libropt.so did not include any files in the test directory.
This is because users were supposes to define their own problem and write their own test file in their language. This is not the situation in the python interface. So the test files need be included. Therefore, the command "$(CPPS)" should be added to the libropt.so option. See line 218 for an example.

[luk-f-a]

Apparently it's not just the test files that are missing. Is it possible $(MANIFOLDS) $(OTHERS) $(PROBLEMS) are actually never defined so when used for libropt.so they are empty? I am not experience with make files but I cannot find where they are defined.
This would also be solved by adding $(CPPS) to the libropt.so option.

[whuang08]

Thank you for pointing this out. I just updated the Makefile.