README example throws error

[EthanHolleman]

Summary

Hi y'all, currently trying to get DrTransformer up and running but using the example code from the README and the fasta files from examples/ is throwing TypeError: in method 'varArrayShort___getitem__'

Programming environment

• Ubuntu 22.04 LTS
• DrTransformer installed into fresh conda env using the command provided by the README
• # INFO - DrTransformer-v1.0: RNA folding kinetics during transcription.

Shell commands and complete error message

Running cat ABCD.fa | DrTransformer --name ABCD gives the error message below. ABCD.fa is taken from the examples folder.

  11    1 ...........   0.00  [1.0000 -> 1.0000] ID = 0
Traceback (most recent call last):
  File "/home/ethollem/anaconda3/envs/drTrans/bin/DrTransformer", line 10, in <module>
    sys.exit(main())
  File "/home/ethollem/anaconda3/envs/drTrans/lib/python3.10/site-packages/drtransformer/drtransformer.py", line 419, in main
    nn, on, prep = TL.expand(args.profile)
  File "/home/ethollem/anaconda3/envs/drTrans/lib/python3.10/site-packages/drtransformer/landscape.py", line 268, in expand
    ndata, edata = neighborhood_flooding((fseq, md, fpwm), ndata, gedges, tedges = edata, minh = minh)
  File "/home/ethollem/anaconda3/envs/drTrans/lib/python3.10/site-packages/drtransformer/rnafolding.py", line 789, in neighborhood_flooding
    for (ss2, en2, ssB, enB, ss1, en1) in edge_flooding(fp, s2, s1, 
  File "/home/ethollem/anaconda3/envs/drTrans/lib/python3.10/site-packages/drtransformer/rnafolding.py", line 701, in edge_flooding
    path, barrier = findpath_split(seq, s1, s2, md, fpwm = fpwm)
  File "/home/ethollem/anaconda3/envs/drTrans/lib/python3.10/site-packages/drtransformer/rnafolding.py", line 562, in findpath_split
    common_basepairs(pt1, pt2)):
  File "/home/ethollem/anaconda3/envs/drTrans/lib/python3.10/site-packages/drtransformer/rnafolding.py", line 486, in common_basepairs
    return iter((i, j1) for i, (j1, j2) in enumerate(zip(pt1[1:], pt2[1:]), 1) 
  File "/home/ethollem/anaconda3/envs/drTrans/lib/python3.10/site-packages/RNA/RNA.py", line 1713, in __getitem__
    return _RNA.varArrayShort___getitem__(self, i)
TypeError: in method 'varArrayShort___getitem__', argument 2 of type 'int'

This same error is also thrown for my own RNA sequences.

Kind of scratching my head at this one, happy to provide additional info if it would be helpful. Thanks and have a good one!

[bad-ants-fleet]

Hi Ethan,

I get the same error when using the latest ViennaRNA package version. Do you use the conda installation? If so, everything should work if you rollback to:

conda install 'viennarna==2.5.1'

I'll have to check with @RaumZeit what the problem is. This may take a few weeks though.

[EthanHolleman]

Sounds good, I'll roll back ViennaRNA and report back!

Update: @bad-ants-fleet your suggestion seems to have worked perfectly. Rolled back to 2.5.1 and same commands on same input fasta ran without issue. Seems like the most recent ViennaRNA breaks something in Dr. T ...

Here is my conda environment yml file for reference exported with the --from-history and some grepping to remove all the extra nonsense conda adds to these.

name: drTransformer
channels:
  - conda-forge
  - bioconda
  - defaults
dependencies:
  - drtransformer
  - viennarna==2.5.1

Maybe add a note to the README in the meantime :)

[RaumZeit]

Hi @bad-ants-fleet @EthanHolleman,

when I read this bug report I already had a vague idea on what might be going wrong here and today I found some time to dig into this issue a little further. So, basically, what has changed with v2.6 of the ViennaRNA Package is the handling of the ptable data structure in the Python interface. Before, those were always actual Python lists with copies of the underlying C data. Since v2.6 the data types only shadow the underlying C data structure but still behave like lists. This has the advantage that no data needs to be converted and copied over and over when underlying C functions are called from within Python.
Unfortunately, I only implemented the __getitem__ and __setitem__ methods, and thought that the remaining functionality of list-like behavior would be handled by Python. But it seems that we also need to explicitly handle slice objects to actually support slicing the ptable.
I started implementing this extension but it might take another week or two until this is finished and compatibility with v2.6 is restored. I'm on vacation right now but will be back next week...

Sorry for the inconveniences!
Best,
Ronny

[RaumZeit]

This should be fixed with v2.6.4 of the ViennaRNA Package...

[julia-gro]

Hi,

I still seem to be getting a related error with any version of ViennaRNA (i tried both 2.6.4 and 2.7.0):

Traceback (most recent call last):
  File "<frozen runpy>", line 198, in _run_module_as_main
  File "<frozen runpy>", line 88, in _run_code
  File "C:\Users\CookieCommander\Downloads\tmp\venv\Scripts\DrTransformer.exe\__main__.py", line 7, in <module>
  File "C:\Users\CookieCommander\Downloads\tmp\venv\Lib\site-packages\drtransformer\drtransformer.py", line 419, in main
    nn, on, prep = TL.expand(args.profile)
                   ^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\CookieCommander\Downloads\tmp\venv\Lib\site-packages\drtransformer\landscape.py", line 268, in expand
    ndata, edata = neighborhood_flooding((fseq, md, fpwm), ndata, gedges, tedges = edata, minh = minh)
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\CookieCommander\Downloads\tmp\venv\Lib\site-packages\drtransformer\rnafolding.py", line 789, in neighborhood_flooding
    for (ss2, en2, ssB, enB, ss1, en1) in edge_flooding(fp, s2, s1,
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\CookieCommander\Downloads\tmp\venv\Lib\site-packages\drtransformer\rnafolding.py", line 701, in edge_flooding
    path, barrier = findpath_split(seq, s1, s2, md, fpwm = fpwm)
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\CookieCommander\Downloads\tmp\venv\Lib\site-packages\drtransformer\rnafolding.py", line 562, in findpath_split
    common_basepairs(pt1, pt2)):
    ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\CookieCommander\Downloads\tmp\venv\Lib\site-packages\drtransformer\rnafolding.py", line 486, in common_basepairs
    return iter((i, j1) for i, (j1, j2) in enumerate(zip(pt1[1:], pt2[1:]), 1)
                                                         ~~~^^^^
  File "C:\Users\CookieCommander\Downloads\tmp\venv\Lib\site-packages\RNA\RNA.py", line 1921, in __getitem__
    return _RNA.varArrayShort___getitem__(self, *args)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
TypeError: Wrong number or type of arguments for overloaded function 'varArrayShort___getitem__'.
  Possible C/C++ prototypes are:
    var_array< short >::__getitem__(int) const
    var_array< short >::__getitem__(SWIGPY_SLICEOBJECT *)

I did however use pip instead of conda to set up my virtual environment. Here are my installed packages:

Package         Version
--------------- -----------
contourpy       1.3.2
cycler          0.12.1
drtransformer   1.0
fonttools       4.58.4
kiwisolver      1.4.8
matplotlib      3.10.3
numpy           2.3.1
packaging       25.0
pillow          11.2.1
pip             22.3.1
pyparsing       3.2.3
python-dateutil 2.9.0.post0
six             1.17.0
ViennaRNA       2.6.4 

Workaround
For now I used a patch of replacing the ptable slices with list comprehensions. [pt[i] for i in range(1, len(pt))].

I think it has got something to do with __getitem__ overloading, hope this helps :)

[bad-ants-fleet]

Hi Julia, thanks for pointing this out! Last time I checked this error only occurred when the package is installed using pip. I'm over my head in other things right now, but I'm glad you found a workaround. Feel free to submit a pull request, otherwise I'll merge it next time I work with DrTrafo again.

[RaumZeit]

Hi!

It took a long time, but the problem is solved with v2.7.1 of the ViennaRNA package. The issue was that v2.7.0 and possibly older versions of the Python bindings available at PyPI were build with an older version of swig that did not properly added all the methods required for slicing our C array we use for the pair tables...
Anyhow, changing the dependency to v2.7.1 will solve the problem!

Cheers