Interface hangs after pressing button

[Integralist]

Hi Freyta.

This looks great, but I noticed that the UI just hangs when you press the button to download the video. Specifically it gets up to Grabbed the m3u8 file, but after that point it hangs.

Not sure how best to debug it.

If I print the account before calling account.down it returns...

{
  '_session': <requests.sessions.Session object at 0x110f5c710>, 
  'user': '...', 
  'password': '...', 
  'logged_in': True, 
  'authorisationToken': '...', 
  'refreshToken': '...'
}

It's at that point the code stops...

self._session.get(link).content 

A URL is generated but it seems the getting of the contents fails?

The URL is...

https://dve-streams.akamaized.net/...

I'm logged into the network with a web browser and attempt to reach that stream URL but get a 403 Forbidden.

[Integralist]

Note: streamlink have a similar issue currently...

streamlink https://watch.wwe.com/episode/NXT-UK-TakeOver-Cardiff-104499 best --loglevel debug
[cli][debug] OS:         Windows 10
[cli][debug] Python:     3.6.6
[cli][debug] Streamlink: 1.2.0+17.g7944fab
[cli][debug] Requests(2.22.0), Socks(1.7.0), Websocket(0.56.0)
[cli][info] Found matching plugin wwenetwork for URL https://watch.wwe.com/episode/NXT-UK-TakeOver-Cardiff-104499
[cli][debug] Plugin specific arguments:
[cli][debug]  (email)
[cli][debug]  (password)
[plugin.wwenetwork][debug] Attempting login as 
[plugin.wwenetwork][debug] API request: POST https://dce-frontoffice.imggaming.com/api/v2/login
[plugin.wwenetwork][debug] Searching for content ID
[plugin.wwenetwork][debug] Loading page config
[plugin.wwenetwork][debug] Found content ID: vod/90904
[plugin.wwenetwork][debug] API request: GET https://dce-frontoffice.imggaming.com/api/v2/stream/vod/90904
[plugin.wwenetwork][debug] API request: GET https://dve-api.imggaming.com/v/90904?customerId=16&auth=d7d3463e7732395a4b06e053bf252714&timestamp=1567868049926
[utils.l10n][debug] Language code: en_US
error: Unable to open URL: https://dve-streams.akamaized.net/ns-003/MjAxOQ/OQ/MQ/MTQx-MTEz_/SE-xT_/MQ/master.m3u8?hdnea=exp=1567868060~acl=/ns-003/MjAxOQ/OQ/MQ/MTQx-MTEz_/SE-xT_/MQ/*~id=85baf02b-f69c-4c66-8db5-e3ed95d25ddb~hmac=de8c5f87bbf6ba38fe4ee6322ccf074c7f20337c9ba063f59437474f2ddf87a6&cId=16&vId=90904&cc=US&rId=&ua=TW96aWxsYS81LjAgKFdpbmRvd3MgTlQgMTAuMDsgV2luNjQ7IHg2NCkgQXBwbGVXZWJLaXQvNTM3LjM2IChLSFRNTCwgbGlrZSBHZWNrbykgQ2hyb21lLzc1LjAuMzc3MC4xNDIgU2FmYXJpLzUzNy4zNg%3D%3D&duuid=85baf02b-f69c-4c66-8db5-e3ed95d25ddb (HTTPSConnectionPool(host='dve-streams.akamaized.net', port=443): Read timed out.)

[freyta]

I'll have a look into this hopefully during the week. I think a command line version of this would be easier for me to make, at least, I have a working version already.

[Integralist]

Ah yeah a CLI version would be really nice! Thanks again for taking the time to look at this.

[freyta]

Hey.
Just an update, I seem to be having a problem with youtube-dl not parsing the m3u8 file properly. It's giving me a garbage response instead of text. Here's an example:

���#�y���O�e9f�ɩ_��0Ic;����aa�c��{cYy5�&��$CU���=5��~�����|Ѵ]�=���f�xk����� �I�t���Fgߗ���0m`

If I can sort this out, I'll give you an update.

[Integralist]

Oh nice! Thanks for the update 👍

I was also trying to find a spare 5 minutes to help the streamlink project who seem to have the same issue (without much success)... streamlink/streamlink#2621 (comment)

[freyta]

I'm fairly sure I can download videos using ffmpeg but it was much slower compared to youtube-dl from memory.

I'll hopefully upload that version today/tonight if it works still

[Integralist]

Awesome. Sounds good!

[freyta]

https://gist.github.com/freyta/6aebc60cf372b77f4387287d2e9ac5bb

I haven't got around to tidying it up completely, but the ffmpeg command works. You can choose to download a specific stream too (i.e. 1080p or 480p) if you know what it is (use ffprobe to figure it out). https://trac.ffmpeg.org/wiki/Map That link will help you with it.

Let me know if you need more help

edit: To use it you need to call it from the command line like this: python3 main.py -t /episode/Superstars-6345. python3 main.py -t episode/Superstars-6345 also works. You will need to uncomment the ffmpeg line and change it so that it uses subprocess.call('ffmpeg command', shell=True) , just like the youtube-dl ones below it.

[Integralist]

AMAZING! Thank you so much for taking the time to get this working. When I get a spare moment I'll try and clean-up the code myself (and in the process hopefully understand how it works too).

[freyta]

Hey.

I've been mucking around a couple of times now, and I think they are restricting who/what can download the m3u8 file. If i use Python requests it gives me gibberish back, just like if i was to use youtube-dl. But if I use wget to download the m3u8 playlist it works fine.. I'm pretty sure youtube-dl uses Python requests that's why it is failing - why it fails, I'm not quite sure.. I still need to figure that one out.

[Integralist]

Ah interesting. I guess you could spoof that value maybe to put their validation off the scent 🤔 I imagine they're looking at the user-agent

[Integralist]

@freyta one thing I noticed was that an episode such as https://watch.wwe.com/episode/Brutus-The-Barber-Beefcake-102081 causes a KeyError...

Traceback (most recent call last):
  File "main.py", line 53, in <module>
    video_link = account.get_video_info(EPISODE)
  File "/Users/markmcdonnell/Code/Python/python3_from_brew/wwedownloader/wwe.py", line 90, in get_video_info
    if entry["customFields"]["EventStyle"] == "Episodic":
KeyError: 'EventStyle'

I've not dived into it but I imagine the EventStyle field will likely have a corresponding value that would need to be checked for.

Same for other shows such as https://watch.wwe.com/original/The-Monday-Night-War-8489

[Integralist]

Ah, I've just come back to look at the code and it's nothing of the sorts, it's just the code presumes the key exists for the sake of generating a filename. So I fix it by using .get instead of direct bracket access...

- if entry["customFields"]["EventStyle"] == "Episodic":
+ if entry["customFields"].get("EventStyle") == "Episodic":

[Integralist]

@freyta ok so that previous comment I made fixed the issue with /episode requests but not /original or /program...

$ python3 main.py -t https://watch.wwe.com/program/Heartbreak-and-Triumph-the-Shawn-Michaels-Story-12256

Logged in
Traceback (most recent call last):
  File "main.py", line 53, in <module>
    video_link = account.get_video_info(EPISODE)
  File "/Users/markmcdonnell/Code/Python/python3_from_brew/wwedownloader/wwe.py", line 88, in get_video_info
    entry = api_link["entries"][0]["item"]
KeyError: 'entries'

The api_link is actually assigned {'message': 'Page not found'}.

I've not had time to dig into that. Maybe you have some ideas on what might be the problem there? Let me know if you do.