Crash on screenlock

[pigmonkey]

Using the latest git version, py3status seems to be unable to recover from a screen lock. The bar runs fine, but if I lock my screen with i3lock and then unlock, the bar is replaced by the i3 error Error: status_command process exited unexpectedly (exit 1). Oddly I can still see the py3status process running via ps.

This does not happen in 2.9.

[pydsigner]

I've had this exact same issue on master for a while now.

[ultrabug]

I lock my screen omgwhat times a day and didn't encounter this problem... BUT I get the same problem on session login !

I will investigate this further, this is a blocker on the release clearly.

Thanks

[tobes]

:( I can't reproduce this via i3lock or by starting a session. My though is it could be related to #253 as signaling is probably involved here.

could someone see if the following commits are good/bad

# commit before signal changes
 git checkout ad8091d8b621d36

# signal changes
 git checkout 09ec0aa355239a

for what it's worth:
I'm on fedora 23, i3 v4.12, i3lock v2.7

[ultrabug]

For my crash, I think it's due to the xrandr module command returning badly on login :

May 18 17:51:14 yazd __init__.py: command "xrandr --output eDP1 --auto --pos 0x0 --rotate normal --output DP1 --auto --right-of eDP1 --rotate normal --output DP2 --off --output HDMI1 --off --output HDMI2 --off --output VIRTUAL1 --off" exit code -12
May 18 17:51:14 yazd __init__.py: Runtime error (TypeError) module.py line 114.
May 18 17:51:14 yazd __init__.py: Runtime error (TypeError) module.py line 114. Please try to fix this and reload i3wm (Mod+Shift+R)
May 18 17:51:14 yazd __init__.py: lock cleared, exiting

My bar is then not updated and a few seconds later I get the status_command error displayed.

[tobes]

Thanks I think I'll be able to trigger it now. Shouldn't be too hard to fix.

[tobes]

@pigmonkey @pydsigner can you see if this is now fixed for you in master

[pigmonkey]

Negative. Same error after screenlock.

[pydsigner]

Not working here either.

[pydsigner]

Interestingly, turning --debug on fixed the issue and turning it off brought it back.

[pydsigner]

So here's all the error I can gather with debug off:

May 19 12:06:38 localhost py3status[10361]: py3status: I3status died horribly. Please try to fix this and reload i3wm (Mod+Shift+R)

[tobes]

Interestingly, turning --debug on fixed the issue and turning it off brought it back.

odd the only code I can see that really does anything is

        # suppress modules' ouput wrt issue #20
        if not self.config['debug']:
            sys.stdout = open('/dev/null', 'w')
            sys.stderr = open('/dev/null', 'w')

but that should in theory help the issue.

if you run it with --standalone does that fix things? At least that would help narrow things down.

[tobes]

@pigmonkey does running with --debug fix things for you?

[tobes]

@pydsigner

This looks like what is killing things https://github.com/ultrabug/py3status/blob/master/py3status/core.py#L579

If the main loop in run() dies then we trigger the error, but why are we dying? It'd be interesting to know what happens if we just comment out the break has i3status actually died or is the thread just stalled for some reason?

[pydsigner]

Hmm that could be possible, also I went away from my PC for ten minutes with it on --debug and it crashed again when logging in.

[tobes]

@pigmonkey @pydsigner another attempt to fix this now in master

[pydsigner]

Current master seems to be working better now. The first fix attempt just left me with cascading I3Status failure nagbars.

[pydsigner]

Hmm nope, happened again.

[tobes]

@pydsigner hmm, what i3status modules do you use? Also the patch just keeps it alive 10 times then lets it die

[ultrabug]

Yeah since it's quite a blocker, it would be great to get deeper in the debug if possible. Like, running i3status alone instead of py3status using the /tmp/pystatus_ config generated (or share it with us if possible?)

[tobes]

Yeah since it's quite a blocker, it would be great to get deeper in the debug if possible. Like, running i3status alone instead of py3status using the /tmp/pystatus_ config generated (or share it with us if possible?)

@pydsigner @pigmonkey any more info you can share with us on this?

[pydsigner]

tmp_i3status.txt

[pigmonkey]

Sorry for the delay.

Using the latest master, I no longer experience the crash when my status command is ~/.virtualenvs/py3status/bin/py3status (I'm keeping the git version in a virtualenv). I tried locking and unlocking my screen 12 times in a row and it worked fine.

I only switched to py3status a few weeks ago. I'm still using wnot for weechat notifications. wnot is supposed to be used with i3status like so:

bar {
    status_command i3status | wnot_cli --i3status
    ...
}

When I moved to py3status stable (2.9). I simply swapped out i3status for py3status.

bar {
    status_command py3status | wnot_cli --i3status
    ...
}

This is what I'm now using on a regular basis. It works fine. However, when I try to use the latest master with wnot, I still experience the crash after unlocking my screen.

bar {
    status_command ~/.virtualenvs/py3status/bin/py3status | wnot_cli --i3status
    ...
}

This only happens when using py3status 3.x and wnot in conjuction.

Now that I'm using py3status I could probably write a module to replace wnot, but wnot does work well and it was nice that I could continue using it with py3status 2.x. I'd rather continue using wnot with py3status 3.x and not put any more thought into the problem of weechat notifications. :)

[tobes]

This only happens when using py3status 3.x and wnot in conjuction.

This makes sense. PR #266 changed things so that i3bar should send us a SIGUSR2 rather than a SIGSTOP so that we could 'suspend' py3status. I suspect that wnot_cli cannot process this and so dies causing the error you see.

As I don't have wnot_cli could you confirm that running with the good py3status 2.9 wnot_cli setup and then running killall -s SIGUSR2 wnot_cli causes the crash.

From what I can see the problem is that wnot_cli is consuming the py3status output and then adding itself into the output before sending it to i3bar. The reason the py3status uses SIGUSR2 rather than SIGSTOP is because some modules using i3 ipc caused i3 to crash hard see #253.

So as far as fixing this the options are.

• Get wnot_cli to accept SIGUSR2 (or whatever is specified by the json it receives from py3status/i3status). This is the real solution but might take time to get implemented.

  The initial response from py3status (master) is {"click_events": true, "stop_signal": 12, "version": 1} in 2.9 it was {"click_events": true, "version": 1} wnot_cli needs to catch and deal with the 'stop_signal' when i3bar sends it. py3status would need to receive this signal to function correctly.

  As you are affected and have history with the project, I'll leave creating an issue for wnot to you but if you or the author need any help then let me know.

• move wnot_cli into a module. A quick and dirty way might be to use the external_script module but you would lose some functionality.

  order += external_script wnot
  
  external_script wnot {
  color = "#00FF00"
  format = "{output}"
  script_path = "wnot_cli --format <FORMAT>"
  }
  

  It would be quite simple to create a wnot py3status module that wrapped calls to wnot_cli --format <FORMAT> and then did nice formatting/color options stuff

Anyhow it's good to get to the bottom of this at last

@pydsigner are you doing anything similar to this in your i3 config? If not then I think your issue may be related to whatever the problem in #169 is and most likely some polling issue.

[ultrabug]

I still think that there's something wrong with this SIGUSR2 signaling. I finally debugged my xrandr problems to be linked with this too.

The xrandr module is mostly broken by SIGURS2, returning error -12 on some changes (so I have to click twice to switch output) then I see in the logs that i3status dies so I think the two are linked.

If I comment this in the core, all those problems go away

# start our output
        header = {
            'version': 1,
            'click_events': True,
            # 'stop_signal': SIGUSR2,
        }

[tobes]

I still think that there's something wrong with this SIGUSR2 signaling.

hmmm, it does seem to be causing us problems.

Maybe it needs a rethink and we should look at fixes for the modules rather than py3status. Moving from using threading to multiprocessing for them might be a way forward and might keep them alive when we get the SIGSTOP.

[tobes]

@pigmonkey py3status (master) has switched to using SIGTSTP rather than SIGUSR2 could you see if this fixes things with your 'bad' config. Hopefully SIGTSTP is better supported.

Once again thanks for your help in this issue

[pigmonkey]

As I don't have wnot_cli could you confirm that running with the good py3status 2.9 wnot_cli setup and then running killall -s SIGUSR2 wnot_cli causes the crash.

This does result in the crash.

With the good py3status 2.9 and wnot setup, running killall -s SIGTSTP wnot_cli does not result in a crash.

With the latest master and a status command of ~/.virtualenvs/py3status/bin/py3status | wnot_cli --i3status, it does crash after unlocking the screen. This time I get the error Error: status_command process exited unexpectedly (exit 1). Originally it was exit 0.

Now that we know what is going on I can work with wnot to get it to accept whatever signal py3status ends up using.

[tobes]

@pigmonkey

Another option might be to have something like the following as a final pipe. It just pipes everything through but handles the signals.

So py3status | wnot_cli --i3status | python pipe.py

you might need to send signals to wnot_cli too

# pipe.py
from __future__ import print_function

from sys import stdin
from signal import signal, SIGCONT, SIGTSTP

from subprocess import call


def sigtstp(signum, frame):
    cmd = 'killall -{} py3status'.format(SIGTSTP)
    call(cmd.split())


def sigcont(signum, frame):
    cmd = 'killall -{} py3status'.format(SIGCONT)
    call(cmd.split())

signal(SIGTSTP, sigtstp)
signal(SIGCONT, sigcont)

while True:
    print(stdin.readline())

[ultrabug]

@pigmonkey do you think we can close this issue mate ?

[tobes]

Closing as this is an issue in wnot_cli not py3status