-
Notifications
You must be signed in to change notification settings - Fork 0
/
Copy pathindex.xml
241 lines (241 loc) · 40 KB
/
index.xml
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
<?xml version="1.0" encoding="utf-8" standalone="yes"?><rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom"><channel><title>Toto do stuff</title><link>https://blog.totetmatt.fr/</link><description>Recent content on Toto do stuff</description><generator>Hugo -- gohugo.io</generator><language>en</language><lastBuildDate>Thu, 27 Apr 2023 00:00:00 +0100</lastBuildDate><atom:link href="https://blog.totetmatt.fr/index.xml" rel="self" type="application/rss+xml"/><item><title>Network Graph rendering : Isopleths with Gmic</title><link>https://blog.totetmatt.fr/posts/2023-02-26_network-graph-rendering-isopleths-with-gmic/</link><pubDate>Sun, 26 Feb 2023 00:00:00 +0100</pubDate><guid>https://blog.totetmatt.fr/posts/2023-02-26_network-graph-rendering-isopleths-with-gmic/</guid><description><p>Mathieu Jacomy is currently experimenting a type of graph rendering using a technic called <strong>“Hillshading”</strong>, a demo is accessible here <a href="https://observablehq.com/d/7d19c2d05caf9fb2">https://observablehq.com/d/7d19c2d05caf9fb2</a> . The idea of this concept is to add information to enhance the readability of the graph, especially when there is condensed clusters of nodes.</p>
<p>The current script is working mostly in javascript with <strong>D3.js</strong>. I wanted to find a way to do it without any JS, locally in my computer. Then I remember a wonderful tool called GMIC that can do a lot of advanced image processing.</p></description><content><p>Mathieu Jacomy is currently experimenting a type of graph rendering using a technic called <strong>“Hillshading”</strong>, a demo is accessible here <a href="https://observablehq.com/d/7d19c2d05caf9fb2">https://observablehq.com/d/7d19c2d05caf9fb2</a> . The idea of this concept is to add information to enhance the readability of the graph, especially when there is condensed clusters of nodes.</p>
<p>The current script is working mostly in javascript with <strong>D3.js</strong>. I wanted to find a way to do it without any JS, locally in my computer. Then I remember a wonderful tool called GMIC that can do a lot of advanced image processing.</p>
<p>To simplify the things, we are going to remove the shading part of the hillshading, which on my opinion isn’t the critical part of the entire process described by Mathieu Jacomy. At the end, hillshading are just shaded <strong>Isopleths</strong> , the line you see in a map that indicate the altitude. I should be enough for the scope of the script we want to acheive.</p>
<p>To use the script we need to have 2 applications :</p>
<ul>
<li><strong>GMIC</strong>, we need the CLI version of the tool. It’s available for Windows and Linux.</li>
<li><strong>Gephi</strong> , for generating graph.</li>
</ul>
<h2 id="export-network-graph">Export Network Graph</h2>
<p>Take any network graph you have, the final effect will wokr better on graph that has a lot of clusters.</p>
<p>When you’re happy with your spatialisation in preview, we need to export in PNG 2 files with the following configuration:</p>
<ul>
<li><strong>background.png</strong> where only the Nodes are rendered, use the Preview Settings to remove the node labels,the edges and the edge labels. Export this file with a certains option : 4096x4096, no transparant background, 0% margin</li>
<li><strong>foreground.png</strong> where you can render the node and the edges, (label might be rendered, but might occur small issue later, I will come back). Export this file with a certains option : 4096x4096, transparant background, 0% margin.</li>
</ul>
<p><img src="https://blog.totetmatt.fr/posts/2023-02-26_network-graph-rendering-isopleths-with-gmic/background-md.png" alt="">
<img src="https://blog.totetmatt.fr/posts/2023-02-26_network-graph-rendering-isopleths-with-gmic/foreground-md.png" alt=""></p>
<h2 id="processing-with-gmic">Processing with Gmic</h2>
<p>Let’s concider you exported the png in the same directory, and gmic is accessible on your terminal, run this command :</p>
<div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-shell" data-lang="shell"><span style="display:flex;"><span> gmic.exe background.png fx_stamp<span style="color:#f92672">[</span>-1<span style="color:#f92672">]</span> 1,100,0,30,0,1,1,0,50,50 fx_channel_processing<span style="color:#f92672">[</span>-1<span style="color:#f92672">]</span> 0,0,0,1.22,2,0,100,256,0,0,0,2,0,0,50,50 samj_Colored_Outlines<span style="color:#f92672">[</span>-1<span style="color:#f92672">]</span> 0,0,16,0,2,0,0,0,255 fx_channel_processing<span style="color:#f92672">[</span>-1<span style="color:#f92672">]</span> 0,0,100,0,0,0,100,256,0,1,0,2,0,0,50,50 output<span style="color:#f92672">[</span>-1<span style="color:#f92672">]</span> intermediate.png
</span></span></code></pre></div><p>Quick explanation :</p>
<ul>
<li><strong>fx_stamp</strong>: Convert the image to black and white and reverse it.</li>
<li><strong>fx_channel_processing</strong>: Apply a blur, this somehow simulate a Kernel Density Estimation. To simplify, the blur processing tries to genreate a density approximation on every point of the map.</li>
<li><strong>samj_Colored_Outlines</strong>: Create the isopleths. We could vugalirse saying it’s a quantization of the discrete density aproximation computed on the previous step.</li>
<li><strong>fx_channel_processing</strong>: Convert to black and white image</li>
<li><strong>output</strong>: Save the image to intermediate.png</li>
</ul>
<p><img src="https://blog.totetmatt.fr/posts/2023-02-26_network-graph-rendering-isopleths-with-gmic/intermediate-3-md.png" alt=""></p>
<p>Then run this script :</p>
<div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-shell" data-lang="shell"><span style="display:flex;"><span>gmic.exe intermediate.png foreground.png +channels<span style="color:#f92672">[</span>-1<span style="color:#f92672">]</span> 100% +image<span style="color:#f92672">[</span>0<span style="color:#f92672">]</span> <span style="color:#f92672">[</span>1<span style="color:#f92672">]</span>,0%,0%,0,0,1,<span style="color:#f92672">[</span>2<span style="color:#f92672">]</span>,255 output<span style="color:#f92672">[</span>-1<span style="color:#f92672">]</span> output.png
</span></span></code></pre></div><p>Here the script only compile the <strong>intermediate.png</strong> and <strong>foreground.png</strong> as one final image.</p>
<p><img src="https://blog.totetmatt.fr/posts/2023-02-26_network-graph-rendering-isopleths-with-gmic/cover.png" alt=""></p>
<h2 id="comments">Comments</h2>
<p>The process is still experimental, a lot of things may vary like the export size that asks to do reparametrization of the script. There is also some limitation due to current behaviour of gephi. If the foreground exports with node label, it might generate a image not aligned with the background which break the global effect at the end.</p>
<p>Out of that, the effect works well with networkt that has a certain critical mass of node density. Having the edges on the foreground hides a little bit the iso lines.</p>
<h2 id="some-other-experiments">Some other experiments</h2>
<p><img src="https://blog.totetmatt.fr/posts/2023-02-26_network-graph-rendering-isopleths-with-gmic/world_border_final-md.png" alt=""></p>
<p><img src="https://blog.totetmatt.fr/posts/2023-02-26_network-graph-rendering-isopleths-with-gmic/rfc_final-2-md.png" alt=""></p>
<p><img src="https://blog.totetmatt.fr/posts/2023-02-26_network-graph-rendering-isopleths-with-gmic/miserable_final-md.png" alt=""></p></content></item><item><title>How to Capture your Bonzomatic with FFmpeg</title><link>https://blog.totetmatt.fr/posts/2021-06-05_bonzomatic_ffmpeg/</link><pubDate>Sat, 05 Jun 2021 11:00:00 +0100</pubDate><guid>https://blog.totetmatt.fr/posts/2021-06-05_bonzomatic_ffmpeg/</guid><description><p>Got to work on this website <a href="https://psenough.github.io/shader_summary/">https://psenough.github.io/shader_summary/</a> that try to gather all graphical live coding events performed in the past.</p>
<p>Basically, for people that don’t know, it’s live coding performance done, sometime as a competition, sometime as a jam, where folks create real time graphics stuff.</p>
<p>One of the common tool used is Bonzomatic , it’s a simple application that use OpenGL to render a rectangle that fit to the application screen and then you can live edit the Framgment Shader that determine what should be the color of the pixel.</p></description><content><p>Got to work on this website <a href="https://psenough.github.io/shader_summary/">https://psenough.github.io/shader_summary/</a> that try to gather all graphical live coding events performed in the past.</p>
<p>Basically, for people that don’t know, it’s live coding performance done, sometime as a competition, sometime as a jam, where folks create real time graphics stuff.</p>
<p>One of the common tool used is Bonzomatic , it’s a simple application that use OpenGL to render a rectangle that fit to the application screen and then you can live edit the Framgment Shader that determine what should be the color of the pixel.</p>
<p>Problem was we got a lot of entries but no preview images. Which is quite sad for a graphics discipline.</p>
<p>After spending an afternoon coding into bonzomatic to find a way to export the bufferframe to an image (was almost here, I think I was missing some color format alignment) I thought about maybe a simpler solution using the best tool ever : FFmpeg.</p>
<p>If we look at the website, there is a way (in Windows at least) to capture an application windows (<a href="https://trac.ffmpeg.org/wiki/Capture/Desktop">https://trac.ffmpeg.org/wiki/Capture/Desktop</a>) .</p>
<p>So using this gdigrab format and using the window’s name, you can capture the input like this :</p>
<div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-shell" data-lang="shell"><span style="display:flex;"><span>ffmpeg -f gdigrab -i <span style="color:#e6db74">&#39;title=BONZOMATIC - GLFW&#39;</span> -vframes <span style="color:#ae81ff">1</span> -q:v <span style="color:#ae81ff">2</span> -y snapshot.jpg
</span></span></code></pre></div><p>Some notes :</p>
<ul>
<li>It will also capture the mouse if it’s inside, so be careful (maybe an option)</li>
<li>If you don’t use fullscreen, it will capture only the “content” of the window, not the menu bar. Which mean that if you maximise, the output resolution will be the screen resolution minus the menu bar + other window frame.</li>
<li>You might want to add a -s 1 before the input to let the application start and / or let ffmpeg get warm before starting a record</li>
</ul>
<p>Of course now you can also export as video. Here is an example of a ffmpeg command that render 10 seconds to mp4 :</p>
<div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-shell" data-lang="shell"><span style="display:flex;"><span>ffmpeg -ss <span style="color:#ae81ff">1</span> -t <span style="color:#ae81ff">10</span> -y -framerate <span style="color:#ae81ff">60</span> -f gdigrab -vsync <span style="color:#ae81ff">0</span> -hwaccel cuda -hwaccel_output_format cuda -i <span style="color:#e6db74">&#39;title=BONZOMATIC - GLFW&#39;</span> -c:a copy -c:v h264_nvenc -tune hq -b:v 20M -bufsize 20M -maxrate 50M -qmin <span style="color:#ae81ff">0</span> -g <span style="color:#ae81ff">250</span> -bf <span style="color:#ae81ff">3</span> -b_ref_mode middle -temporal-aq <span style="color:#ae81ff">1</span> -rc-lookahead <span style="color:#ae81ff">20</span> -i_qfactor 0.75 -b_qfactor 1.1 out.mp4
</span></span></code></pre></div><p><em>(I copy pasted some config + blind tweak. Can&rsquo;t really explain the options but was happy with result)</em></p>
<p>I’m using nvidia_env as it’s much faster for encoding and avoid issues I got with the normal libx264.</p>
<p>Need to check more in detail for Sound capture at the same time and see if there is different input format to play with.</p></content></item><item><title>Twitch and FFmpeg and Youtube-dl: Fetch from live stream to local file</title><link>https://blog.totetmatt.fr/posts/2020-10-13_twitch-and-ffmpeg-with-some-youtube-dl-help-fetch-from-live-stream-to-local-file/</link><pubDate>Sat, 03 Oct 2020 11:00:00 +0100</pubDate><guid>https://blog.totetmatt.fr/posts/2020-10-13_twitch-and-ffmpeg-with-some-youtube-dl-help-fetch-from-live-stream-to-local-file/</guid><description><p><em>(Using Windows PowerShell, adapt for UNIX bash shouldn’t be a big issue)</em></p>
<h1 id="record-a-live-stream">Record a live stream</h1>
<p>So something nice with youtube-dl is like you can ask not to download the media but to fetch for the media link :</p>
<div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-shell" data-lang="shell"><span style="display:flex;"><span>&gt;&gt; youtube-dl -g https://www.youtube.com/watch?v<span style="color:#f92672">=</span>RJt01u4yrLQ
</span></span><span style="display:flex;"><span>https://r2---sn-h0jeened.googlevideo.com/videoplayback?expire<span style="color:#f92672">=</span>1<span style="color:#f92672">[</span>...<span style="color:#f92672">]</span>
</span></span></code></pre></div><p>If you use that for a twitch channel that is streaming live, it returns you the HLS stream.</p>
<div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-shell" data-lang="shell"><span style="display:flex;"><span>&gt;&gt; youtube-dl -g https://www.twitch.tv/farore_de_firone
</span></span><span style="display:flex;"><span>https://video-weaver.ber01.hls.ttvnw.net/v1/playlist/CpkEQusnrcdffNI3<span style="color:#f92672">[</span>..<span style="color:#f92672">]</span>MA2c4.m3u8
</span></span></code></pre></div><p>It by default binds to the best quality video, but you can check and select all format available using -F</p></description><content><p><em>(Using Windows PowerShell, adapt for UNIX bash shouldn’t be a big issue)</em></p>
<h1 id="record-a-live-stream">Record a live stream</h1>
<p>So something nice with youtube-dl is like you can ask not to download the media but to fetch for the media link :</p>
<div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-shell" data-lang="shell"><span style="display:flex;"><span>&gt;&gt; youtube-dl -g https://www.youtube.com/watch?v<span style="color:#f92672">=</span>RJt01u4yrLQ
</span></span><span style="display:flex;"><span>https://r2---sn-h0jeened.googlevideo.com/videoplayback?expire<span style="color:#f92672">=</span>1<span style="color:#f92672">[</span>...<span style="color:#f92672">]</span>
</span></span></code></pre></div><p>If you use that for a twitch channel that is streaming live, it returns you the HLS stream.</p>
<div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-shell" data-lang="shell"><span style="display:flex;"><span>&gt;&gt; youtube-dl -g https://www.twitch.tv/farore_de_firone
</span></span><span style="display:flex;"><span>https://video-weaver.ber01.hls.ttvnw.net/v1/playlist/CpkEQusnrcdffNI3<span style="color:#f92672">[</span>..<span style="color:#f92672">]</span>MA2c4.m3u8
</span></span></code></pre></div><p>It by default binds to the best quality video, but you can check and select all format available using -F</p>
<div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-shell" data-lang="shell"><span style="display:flex;"><span>&gt;&gt; youtube-dl -F https://www.twitch.tv/farore_de_firone
</span></span><span style="display:flex;"><span><span style="color:#f92672">[</span>twitch:stream<span style="color:#f92672">]</span> farore_de_firone: Downloading stream GraphQL
</span></span><span style="display:flex;"><span><span style="color:#f92672">[</span>twitch:stream<span style="color:#f92672">]</span> farore_de_firone: Downloading access token JSON
</span></span><span style="display:flex;"><span><span style="color:#f92672">[</span>twitch:stream<span style="color:#f92672">]</span> 39653517372: Downloading m3u8 information
</span></span><span style="display:flex;"><span><span style="color:#f92672">[</span>info<span style="color:#f92672">]</span> Available formats <span style="color:#66d9ef">for</span> 39653517372:
</span></span><span style="display:flex;"><span>format code extension resolution note
</span></span><span style="display:flex;"><span>audio_only mp4 audio only 2k , mp4a.40.2
</span></span><span style="display:flex;"><span>160p mp4 284x160 230k , avc1.4D401F, 30.0fps, mp4a.40.2
</span></span><span style="display:flex;"><span>360p mp4 640x360 630k , avc1.4D401F, 30.0fps, mp4a.40.2
</span></span><span style="display:flex;"><span>480p mp4 852x480 1262k , avc1.4D401F, 30.0fps, mp4a.40.2
</span></span><span style="display:flex;"><span>720p60 mp4 1280x720 3257k , avc1.4D401F, 60.0fps, mp4a.40.2
</span></span><span style="display:flex;"><span>1080p60__source_ mp4 1920x1080 6713k , avc1.64002A, 60.0fps, mp4a.40.2 <span style="color:#f92672">(</span>best<span style="color:#f92672">)</span>
</span></span></code></pre></div><p>You could even only have the audio-stream</p>
<p>And to select it</p>
<div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-shell" data-lang="shell"><span style="display:flex;"><span>youtube-dl -f 160p -g https://www.twitch.tv/farore_de_firone
</span></span></code></pre></div><p>So with this link, you can use ffmpeg to record localy the stream to your computer (and have your own reaplay / VOD without the “disagreement” of Twitch VOD :) )</p>
<div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-shell" data-lang="shell"><span style="display:flex;"><span>ffmpeg -i <span style="color:#e6db74">&#34;</span><span style="color:#66d9ef">$(</span>youtube-dl -f 720p60 -g https://www.twitch.tv/farore_de_firone<span style="color:#66d9ef">)</span><span style="color:#e6db74">&#34;</span> -c copy stream.20201012.mp4
</span></span></code></pre></div><p>And here it’s quite simple “dump” of the running script. Nothing prevent you to add some filters, reencoding that adapt to your needs.</p>
<h1 id="mixing-multiple-stream">Mixing multiple stream</h1>
<p>Let’s have some fun, there is some streamers that plays together on the same game. Usually, you can watch their POV at the sametime with the Twitch Squad mechanism or Multitwitch application. But would it be possible to record a file in such way ?</p>
<p>Actually yes, ffmpeg can take multiple video input and transform it on the fly via the filter complex to render all the video on the same stream.</p>
<p>There is a nice topic on Stackoverflow that explain how to simply stack multiple video : <a href="https://stackoverflow.com/questions/11552565/vertically-or-horizontally-stack-mosaic-several-videos-using-ffmpeg">https://stackoverflow.com/questions/11552565/vertically-or-horizontally-stack-mosaic-several-videos-using-ffmpeg</a></p>
<p>Example merging 2 videos :</p>
<div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-shell" data-lang="shell"><span style="display:flex;"><span>ffmpeg -i <span style="color:#e6db74">&#34;</span><span style="color:#66d9ef">$(</span>youtube-dl -g https://www.twitch.tv/antoinedaniellive<span style="color:#66d9ef">)</span><span style="color:#e6db74">&#34;</span> <span style="color:#ae81ff">\
</span></span></span><span style="display:flex;"><span><span style="color:#ae81ff"></span> -i <span style="color:#e6db74">&#34;</span><span style="color:#66d9ef">$(</span>youtube-dl -g https://www.twitch.tv/soon<span style="color:#66d9ef">)</span><span style="color:#e6db74">&#34;</span> <span style="color:#ae81ff">\
</span></span></span><span style="display:flex;"><span><span style="color:#ae81ff"></span> -filter_complex vstack<span style="color:#f92672">=</span>inputs<span style="color:#f92672">=</span><span style="color:#ae81ff">2</span> <span style="color:#ae81ff">\
</span></span></span><span style="display:flex;"><span><span style="color:#ae81ff"></span> -map 0:a <span style="color:#ae81ff">\
</span></span></span><span style="display:flex;"><span><span style="color:#ae81ff"></span> output.mp4
</span></span></code></pre></div><p>Example merging 4 videos :</p>
<div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-shell" data-lang="shell"><span style="display:flex;"><span>ffmpeg -i <span style="color:#e6db74">&#34;</span><span style="color:#66d9ef">$(</span>youtube-dl -f 160p -g https://www.twitch.tv/antoinedaniellive<span style="color:#66d9ef">)</span><span style="color:#e6db74">&#34;</span> <span style="color:#ae81ff">\
</span></span></span><span style="display:flex;"><span><span style="color:#ae81ff"></span> -i <span style="color:#e6db74">&#34;</span><span style="color:#66d9ef">$(</span>youtube-dl -f 160p -g https://www.twitch.tv/soon<span style="color:#66d9ef">)</span><span style="color:#e6db74">&#34;</span> <span style="color:#ae81ff">\
</span></span></span><span style="display:flex;"><span><span style="color:#ae81ff"></span> -i <span style="color:#e6db74">&#34;</span><span style="color:#66d9ef">$(</span>youtube-dl -f 160p -g https://www.twitch.tv/angledroit <span style="color:#66d9ef">)</span><span style="color:#e6db74">&#34;</span> <span style="color:#ae81ff">\
</span></span></span><span style="display:flex;"><span><span style="color:#ae81ff"></span> -i <span style="color:#e6db74">&#34;</span><span style="color:#66d9ef">$(</span>youtube-dl -f 160p -g https://www.twitch.tv/etoiles<span style="color:#66d9ef">)</span><span style="color:#e6db74">&#34;</span> <span style="color:#ae81ff">\
</span></span></span><span style="display:flex;"><span><span style="color:#ae81ff"></span> -filter_complex <span style="color:#e6db74">&#34;[0:v][1:v][2:v][3:v]xstack=inputs=4:layout=0_0|w0_0|0_h0|w0_h0[v]&#34;</span> -map <span style="color:#e6db74">&#34;[v]&#34;</span> <span style="color:#ae81ff">\
</span></span></span><span style="display:flex;"><span><span style="color:#ae81ff"></span> -map 0:a <span style="color:#ae81ff">\
</span></span></span><span style="display:flex;"><span><span style="color:#ae81ff"></span> -y output.mp4
</span></span></code></pre></div><p>Note : it’s better to stack video vertically. Horizontal stack does works but then some services (like twitter) won’t accept the video because the image ratio will be too extreme.</p>
<p>The -map 0:a here is necessary to select which audio you want to have.</p>
<p>The format mkv also allow to record multiple video stream within one file that you can selected after that :</p>
<div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-shell" data-lang="shell"><span style="display:flex;"><span>ffmpeg -i <span style="color:#e6db74">&#34;</span><span style="color:#66d9ef">$(</span>youtube-dl -g https://www.twitch.tv/alphacast<span style="color:#66d9ef">)</span><span style="color:#e6db74">&#34;</span> <span style="color:#ae81ff">\ </span>
</span></span><span style="display:flex;"><span> -i <span style="color:#e6db74">&#34;</span><span style="color:#66d9ef">$(</span>youtube-dl -g https://www.twitch.tv/colas_bim<span style="color:#66d9ef">)</span><span style="color:#e6db74">&#34;</span> <span style="color:#ae81ff">\
</span></span></span><span style="display:flex;"><span><span style="color:#ae81ff"></span> -i <span style="color:#e6db74">&#34;</span><span style="color:#66d9ef">$(</span>youtube-dl -g https://www.twitch.tv/eventisfr<span style="color:#66d9ef">)</span><span style="color:#e6db74">&#34;</span> <span style="color:#ae81ff">\
</span></span></span><span style="display:flex;"><span><span style="color:#ae81ff"></span> -i <span style="color:#e6db74">&#34;</span><span style="color:#66d9ef">$(</span>youtube-dl -g https://www.twitch.tv/fusiow<span style="color:#66d9ef">)</span><span style="color:#e6db74">&#34;</span> <span style="color:#ae81ff">\
</span></span></span><span style="display:flex;"><span><span style="color:#ae81ff"></span> -map 0:1 -map 1:1 -map 2:1 -map 3:1 -map 0:0 <span style="color:#ae81ff">\ </span>
</span></span><span style="display:flex;"><span> -c copy <span style="color:#ae81ff">\
</span></span></span><span style="display:flex;"><span><span style="color:#ae81ff"></span> out.mkv
</span></span></code></pre></div></content></item><item><title>Extract chapters from Youtube Media</title><link>https://blog.totetmatt.fr/posts/2020-06-26_extract-chapters-youtube-media/</link><pubDate>Fri, 26 Jun 2020 11:00:00 +0100</pubDate><guid>https://blog.totetmatt.fr/posts/2020-06-26_extract-chapters-youtube-media/</guid><description><blockquote>
<p>Update: Since a while, <code>youtube-dl</code> isn&rsquo;t as powerful as it is. Fortunatelly another fork exists : <code>yt-dlp</code> . Use this instead</p></blockquote>
<p>Youtube recently got this “chapter” concept where it fragment a long video with chapters. I think this data might be parsed from the description of the video done, as they already parse any timestamp available for a while now.</p>
<p>Thanks to <code>youtube-dl</code>, we can download thena video and the metadata which now contains this chapter data.</p></description><content><blockquote>
<p>Update: Since a while, <code>youtube-dl</code> isn&rsquo;t as powerful as it is. Fortunatelly another fork exists : <code>yt-dlp</code> . Use this instead</p></blockquote>
<p>Youtube recently got this “chapter” concept where it fragment a long video with chapters. I think this data might be parsed from the description of the video done, as they already parse any timestamp available for a while now.</p>
<p>Thanks to <code>youtube-dl</code>, we can download thena video and the metadata which now contains this chapter data.</p>
<pre tabindex="0"><code class="language-terminal" data-lang="terminal">$ youtube-dl --write-info-json -x --audio-format mp3 https://www.youtube.com/watch?v=HZTStHzWRxM
[youtube] HZTStHzWRxM: Downloading webpage
[info] Writing video description metadata as JSON to: The New Youtube Chapter Timestamp Feature-HZTStHzWRxM.info.json
[download] Destination: The New Youtube Chapter Timestamp Feature-HZTStHzWRxM.webm
[download] 100% of 3.22MiB in 00:00
[ffmpeg] Destination: The New Youtube Chapter Timestamp Feature-HZTStHzWRxM.mp3
Deleting original file The New Youtube Chapter Timestamp Feature-HZTStHzWRxM.webm (pass -k to keep)
</code></pre><p>We will use <a href="https://www.youtube.com/watch?v=HZTStHzWRxM">https://www.youtube.com/watch?v=HZTStHzWRxM</a> as example.</p>
<p>The command above will download the video file, transcode it to mp3 and also download the metadata in a json format. We have now 2 files :</p>
<ul>
<li><code>The New Youtube Chapter Timestamp Feature-HZTStHzWRxM.info.json</code> that contains data</li>
<li><code>The New Youtube Chapter Timestamp Feature-HZTStHzWRxM.mp3</code> that is the media</li>
</ul>
<p><code>jq</code> is a wonderful command line to manipulate json on bash. We can for example get the title of the video like this :</p>
<pre tabindex="0"><code class="language-terminal" data-lang="terminal">$ cat The\ New\ Youtube\ Chapter\ Timestamp\ Feature-HZTStHzWRxM.info.json | jq -r .title | sed -e &#39;s/[^A-Za-z0-9._-]/_/g&#39;
The_New_Youtube_Chapter_Timestamp_Feature
</code></pre><p>The sed here is to make sure we won’t have special characters that might lead to some error later.</p>
<p>The <code>-r</code> on <code>jq</code> indicate to return “raw text”. By default, <code>jq</code> will use some syntax colorization and keep some sepcial character that might leads to some issue.</p>
<p>If available, <code>Youtube-dl</code> info json contains a chapters array that contain all the chapters with their <code>start_time</code> , <code>end_time</code> and <code>title</code> .</p>
<pre tabindex="0"><code class="language-terminal" data-lang="terminal">$ cat The\ New\ Youtube\ Chapter\ Timestamp\ Feature-HZTStHzWRxM.info.json |\
jq -r &#39;.chapters[]&#39;
{
&#34;start_time&#34;: 0,
&#34;end_time&#34;: 17,
&#34;title&#34;: &#34;The new feature&#34;
}
{
&#34;start_time&#34;: 17,
&#34;end_time&#34;: 76,
&#34;title&#34;: &#34;Slow roll-out&#34;
}
{
&#34;start_time&#34;: 76,
&#34;end_time&#34;: 124,
&#34;title&#34;: &#34;How it works&#34;
}
{
&#34;start_time&#34;: 124,
&#34;end_time&#34;: 180,
&#34;title&#34;: &#34;Problems / suggestions for the future&#34;
}
</code></pre><p>The idea now is to use each dict entry here as parameters for ffmpeg to split the media according to the chapters data. As we are in bash, current json representation will be quite hard to use it like that, so we need to transform a little bit the representation here to use the output of <code>jq</code> in a pipe and in <code>xarg</code>s.</p>
<p>What also we need to take into consideration is that ffmpeg can split a media by giving the option <code>-ss</code> to know where to start and <code>-t</code> to know the duration of the cut, not the end time. As the information on the json gives us a start and end time, we need to perfom a simple substraction to have the start time and the duration.</p>
<pre tabindex="0"><code class="language-terminal" data-lang="terminal">$ cat The\ New\ Youtube\ Chapter\ Timestamp\ Feature-HZTStHzWRxM.info.json |\
jq -r &#39;.chapters[] | .start_time,.end_time-.start_time,.title &#39; |\
sed &#39;s/&#34;//g&#39;
0
17
The new feature
17
59
Slow roll-out
76
48
How it works
124
56
Problems / suggestions for the future
</code></pre><p>Thanks to <code>jq</code>, we can perfom simple math operation directly on the command to compute the duration. sed here again is only for cleaning up special characters.</p>
<p>Now, we can pipe the wonderful xargs to use the output as parameter and trigger a <code>ffmpeg</code> command</p>
<pre tabindex="0"><code class="language-terminal" data-lang="terminal">$ cat The\ New\ Youtube\ Chapter\ Timestamp\ Feature-HZTStHzWRxM.info.json|\
jq -r &#39;.chapters[] | .start_time,.end_time-.start_time,.title &#39; |\
sed -e &#39;s/[^A-Za-z0-9._-]/_/g&#39; |\
xargs -n3 -t -d&#39;\n&#39; sh -c &#39;ffmpeg -y -ss $0 -i &#34;The New Youtube Chapter Timestamp Feature-HZTStHzWRxM.mp3&#34; -t $1 -codec:a copy &#34;$2.mp3&#34;&#39;
</code></pre><ul>
<li><code>-n3</code> indicate to take parameters 3 by 3*</li>
<li><code>-t</code> is only to debug as it will print each command xargs will execute</li>
<li><code>-d'\n'</code> indicate that parameters are separated by <code>\n</code></li>
</ul>
<p>What is cool is that we could potentially parallelize the process here by adding to <code>xargs</code> the parameter <code>-P X</code> to run the multiple ffmpeg invokation in parallel.</p>
<p>On ffmpeg side, nothing tremendous :</p>
<ul>
<li><code>-ss</code> and <code>-t</code> has been already explain as start time and duration,</li>
<li><code>-codec:a</code> copy indicate that we keep everything same as the original file in terms of codec, so no re-encoding for the output file, which means it’s going fast</li>
<li><code>-y</code> to avoid prompt and force override of existing output file</li>
</ul>
<p>That works quite well. It might be possible to fully one line it, but let’s put a proper script to ease the usage of this.</p>
<div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-shell" data-lang="shell"><span style="display:flex;"><span><span style="color:#75715e">#!/bin/sh
</span></span></span><span style="display:flex;"><span><span style="color:#75715e"></span>set -x
</span></span><span style="display:flex;"><span>
</span></span><span style="display:flex;"><span><span style="color:#75715e">#Download media + metadata</span>
</span></span><span style="display:flex;"><span>youtube-dl --write-info-json -x --audio-format mp3 -o <span style="color:#e6db74">&#34;tmp_out.%(ext)s&#34;</span> $1
</span></span><span style="display:flex;"><span>
</span></span><span style="display:flex;"><span><span style="color:#75715e"># Maybe a way to get the file name from previous function</span>
</span></span><span style="display:flex;"><span>INFO<span style="color:#f92672">=</span><span style="color:#e6db74">&#34;tmp_out.info.json&#34;</span>
</span></span><span style="display:flex;"><span>AUDIO<span style="color:#f92672">=</span><span style="color:#e6db74">&#34;tmp_out.mp3&#34;</span>
</span></span><span style="display:flex;"><span>echo :: $INFO $AUDIO ::
</span></span><span style="display:flex;"><span><span style="color:#75715e"># Fetch the title</span>
</span></span><span style="display:flex;"><span>TITLE<span style="color:#f92672">=</span><span style="color:#66d9ef">$(</span>cat <span style="color:#e6db74">&#34;</span>$INFO<span style="color:#e6db74">&#34;</span> | jq -r .title | sed -e <span style="color:#e6db74">&#39;s/[^A-Za-z0-9._-]/_/g&#39;</span> <span style="color:#66d9ef">)</span>
</span></span><span style="display:flex;"><span> <span style="color:#75715e"># ^--- Remove all weird character as we want to use it as filename</span>
</span></span><span style="display:flex;"><span><span style="color:#75715e"># We will put all chapter into a directory</span>
</span></span><span style="display:flex;"><span>mkdir <span style="color:#e6db74">&#34;</span>$TITLE<span style="color:#e6db74">&#34;</span>
</span></span><span style="display:flex;"><span>
</span></span><span style="display:flex;"><span><span style="color:#75715e"># Chapterization</span>
</span></span><span style="display:flex;"><span>cat <span style="color:#e6db74">&#34;</span>$INFO<span style="color:#e6db74">&#34;</span> |<span style="color:#ae81ff">\
</span></span></span><span style="display:flex;"><span><span style="color:#ae81ff"></span>jq -r <span style="color:#e6db74">&#39;.chapters[] | .start_time,.end_time-.start_time,.title &#39;</span> |<span style="color:#ae81ff">\
</span></span></span><span style="display:flex;"><span><span style="color:#ae81ff"></span>sed -e <span style="color:#e6db74">&#39;s/[^A-Za-z0-9._-]/_/g&#39;</span> |<span style="color:#ae81ff">\
</span></span></span><span style="display:flex;"><span><span style="color:#ae81ff"></span>xargs -n3 -t -d<span style="color:#e6db74">&#39;\n&#39;</span> sh -c <span style="color:#e6db74">&#34;ffmpeg -y -ss \$0 -i \&#34;</span>$AUDIO<span style="color:#e6db74">\&#34; -to \$1 -codec:a copy -f mp3 \&#34;</span>$TITLE<span style="color:#e6db74">/\$2.mp3\&#34;&#34;</span>
</span></span><span style="display:flex;"><span>
</span></span><span style="display:flex;"><span><span style="color:#75715e">#Remove tmp file</span>
</span></span><span style="display:flex;"><span>rm tmp_out*
</span></span></code></pre></div><p>The script file here : <a href="https://gist.github.com/totetmatt/b4bf50c62642e5a9e1bf6365a47e19c6">https://gist.github.com/totetmatt/b4bf50c62642e5a9e1bf6365a47e19c6</a></p>
<p>No big change on the global approach just something to becareful : Yes, there is a hell quote escape game to play and it might not be pleasant ….</p>
<p>To explain the last part, as far as I understand it, the string will be evaluated multiple time :</p>
<ul>
<li>First time will be at “script level”, so it will replace any $VARIABLE present in the script like <code>$AUDIO</code> and <code>$TITLE</code></li>
<li>Second time will be at <code>xargs</code> / <code>sh -c</code> invokation where then it’s possible to use <code>$0</code> <code>$1</code> and <code>$2</code>. But if we don’t escape it first, theses variables will be evaluated at the first round, that’s why we need to backslash it <code>\$0, \$1, \$2</code>.</li>
</ul>
<p>You can see the result of the string after the 1st evaluation thanks to the <code>-t</code> option of <code>xargs</code> :</p>
<pre tabindex="0"><code class="language-terminal" data-lang="terminal">sh -c &#39;ffmpeg -y -ss $0 -i &#34;The New Youtube Chapter Timestamp Feature-HZTStHzWRxM.mp3&#34; -to $1 -codec:a copy -f mp3 &#34;The_New_Youtube_Chapter_Timestamp_Feature/$2.mp3&#34;&#39; 124 56 Problems___suggestions_for_the_future
</code></pre><p>There might be other and better way to deal wih the args parsing, the string escape and the string cleanup, but current solution works enough :)</p></content></item></channel></rss>