New CUDA version #50
Replies: 13 comments 46 replies
-
|
I did not have any trouble in installing or starting the process of generating a transcript (CPU usage remained below 20%). The file I tried was not in English and I faced the issue you listed in known issues (loop of repeating text) after the 10th minute in a 30 minute audio file. |
Beta Was this translation helpful? Give feedback.
-
|
Thank you for testing, sounds promising! |
Beta Was this translation helpful? Give feedback.
-
Any special specific install instructions on how to install and try this version? |
Beta Was this translation helpful? Give feedback.
-
|
small addition: Please change "config.yml" only when noScribe is closed. |
Beta Was this translation helpful? Give feedback.
-
|
@Lod3: could you share the output of your |
Beta Was this translation helpful? Give feedback.
-
|
@gernophil thanks, I figured something was wrong with the CUDA drivers so I reinstalled them on my desktop. Seems to work now |
Beta Was this translation helpful? Give feedback.
-
|
Nice! If the animated gif shows the transciption in realtime, that's pretty fast - especially given the fact that you use the 'precise' model and a mid-level graphics card.
The "config.yml" is in
Could you please make a simple writeup on how to find and install these drivers for a particular card? I would like to integrate that in the readme for other users in the future. Nice to hear that this is working so well for your interesting application. If you have feature ideas, feel free to create an issue! |
Beta Was this translation helpful? Give feedback.
-
|
I think, if you simply install the latest Nvidia drivers for you card it should be fine. CUDA should be bundled with them. And using nvidia-smi you can check your cuda version. |
Beta Was this translation helpful? Give feedback.
-
|
@VBodrov reported in another thread:
Interesting. There is a good chance that 4GB of VRAM are not sufficient for the large whisper model. You can try the "fast" setting in noScribe. This will use a much smaller model. The quality is not great. But this would confirm that in fact the size of the model is the problem. |
Beta Was this translation helpful? Give feedback.
-
It's really simple
Done
The gif from earliers is from my desktop this is the result from the laptop on which I tried transcribing earlier with the latest drivers installed
I would still need to install the CUDA drivers like I mentioned above because I just tested it and noscribe on the laptop does not use the GPU when transcribing. I just tested it again. I am going to install the CUDA drivers now and report back :) The laptop config |
Beta Was this translation helpful? Give feedback.
-
|
hmmm maybe cuda its not supported on this GPU 🤔 pitty |
Beta Was this translation helpful? Give feedback.
-
|
Is the first step (speaker identification) using your GPU, at least to some extend? This would suggest that cuda is working and we only need to point faster-whisper to right libraries. |
Beta Was this translation helpful? Give feedback.
-
|
Thanks! Works like charm for me! Die Schatzinsel (342 min) works for me on (But i think, I installed Cuda by myself before on all three Systems) Also diarizing used cuda. Just typical small bugs (setting rights on win11, language on auto -> always translating to english automatically) Speech-Translate (https://github.com/Dadangdut33/Speech-Translate (also faster whisper)) did not work on laptop w/ RTX_A500 |
Beta Was this translation helpful? Give feedback.
-
|
I tested it with 2060 super. It works with both diarization with PyAnnote and whisper. I am running it under widnwos 11 with |
Beta Was this translation helpful? Give feedback.
-
|
it works! I will make some suggestions that will be useful 1.Upload video in batch.
As soon as I have some time to do a comparative test CPU Xeon 14 cores 128 GB RAM vs 3060 12gb THANKS! |
Beta Was this translation helpful? Give feedback.
-
|
Thank you very much for this release ! On my side, it did work better (went from 19mn to transcribe 5mn audio to 4-6mn), and it seems the GPU is working to some extent. Also checked the yml file, everything is correct. I also added the PATH.
Thank you for your help ! |
Beta Was this translation helpful? Give feedback.
-
|
Thanks for testing!
This is quite an improvement so the GPU must work harder than shown in the screenshot. Can you check if it is used during the transcription phase? My guess is that pyannote is not using the GPU all that well, but faster-whisper does. I may need to update pyannote, there is a new version out. |
Beta Was this translation helpful? Give feedback.
-
|
Well using windows task monitor, it stays the same both during annotation and transcription. With windows task monitor during transcription : With HWInfo : Pyannote : Transcription : And the peak use (CUDA) : But as I said, it seems to work randomly. Most of the time the GPU use is close to 0%. Thanks for the answer :) |
Beta Was this translation helpful? Give feedback.
-
|
The transcription is a complex process, consisting of many steps. Only some of them can be accelerated by the GPU, others must run on the CPU. Your imporvements in overall transcription time are quite impressive. I think this is the maximum we can do at the moment. The only option would be to run several instances of noScribe in parallel, like @antonioaraujo664 did above: #50 (comment) At some point, you will run out of GPU-memory though. |
Beta Was this translation helpful? Give feedback.
-
|
The 0.4.5 Cuda Version runs like a charm and the quality of transcription results compete with amberscript. Thanks a lot for this release, the CUDA acceleration gains a big performance boost. The CUDA transcription speeds scales massive with RAM speed/timings/bandwith. System 1: Intel i7-13700k, 16GB DDR5-4400 CL36 RAM (One Module - single channel), NVidia RTX4070 System 2.1: AMD Threadripper1950x, 32GB DDR4-2133 CL15 (Four Modules - quad channel), NVidia RTX4070 System 2.2: AMD Threadripper1950x, 32GB DDR4-3600 CL17 (Four Modules - quad channel), NVidia RTX4070 Maybe this should be mentioned in the recomended hardware section, I've spend some hours to get it why the much recent Intel system is much slower than our old Threadripper workstation. |
Beta Was this translation helpful? Give feedback.
-
|
Interesting observation regarding the RAM speed. I think I will compile a separate document with all the information about hardware, drivers etc. for a fast cuda based system (I've already started to edit my initial post above). Overall, your numbers are very impressive. Is this with speaker detection on? Have you also tried running multiple instances in parallel like antonioaraujo664 (#50 (comment))? BTW, do you plan running noScribe on a server? A collegue from the University of Jena is in the process of developing a client-server version. I hope this will be ready within the the next few months. |
Beta Was this translation helpful? Give feedback.
-
|
Speaker detection was on. [Test 1] running a single instance of noScribe
[Test 2] running 3 multiple instances of noScribe in parallel like antonioaraujo664 (#50 (comment)) Indeed, we're planning to run noscribe on a server, so server/client development would be perfect!
|
Beta Was this translation helpful? Give feedback.
-
|
Thank you for this systematic testing, that's very helpful. This is a little slower than your initial test with the shorter interview suggested, but still a very good result. @eckhrd is the one who is going to work on the client/server version soon. You may ask him if he needs help... |
Beta Was this translation helpful? Give feedback.
-
|
I've done a RAM upgrade on the i7-13070K Workstation Single->Dual Channel, |
Beta Was this translation helpful? Give feedback.
-
|
I didn't know these efficiency cores are a thing with windows, too. For macOS I only use the performance cores, which i get using |
Beta Was this translation helpful? Give feedback.
-
|
Each instance consumes an average of 3.5 GB of VRAM. On the 3060 12gb there are times when the machine freezes with three. Any plans to get batch processing one by one in order? |
Beta Was this translation helpful? Give feedback.
-
|
Yes, batch processing is high on my list. But it may still take a while until it becomes reality. As mentioned above, we are working on a client/server version. This needs some restructuring of the code which will also make batch-processing much easier to implement. We have to take it step by step. |
Beta Was this translation helpful? Give feedback.
-
|
Have you tried to install the Nvidia Geforce "Studio" Driver? These Drivers are more stable than the "Game Ready" Drivers. |
Beta Was this translation helpful? Give feedback.
-
|
Hello! This new version is running super smooth on my computer compared to the previous version of noScribe. Before, noScribe would freeze while it was transcribing and it was really terrifying but it always got the transcription done. I only had that weird repeating word issue happen once. This new version can run in the background while I do other work so big fan! I'm running an old Alienware from 2016 with a GTX 1060. Thanks again for making my PhD life easier!! |
Beta Was this translation helpful? Give feedback.
-
|
Yes! Can confirm 6 gb of vram. |
Beta Was this translation helpful? Give feedback.
-
|
I've gone through a few files now and am running up against the repeating word loop thing way more than with the old version. Any suggestions for adjustment of settings that could help this? Before the word loop the transcription is very good! |
Beta Was this translation helpful? Give feedback.
-
|
Strange that this occurs more often now, it should be absolutely the same. Maybe it just feels more often because the transcription is so much quicker now? |
Beta Was this translation helpful? Give feedback.
-
|
Listened a little more carefully to when the word loop starts because it has been at very specific points in my audio files. It seems that there was some glitchy silence in both of the points where the word loop started. In one instance the silence was maybe about 13ish seconds long. In the other instance there was a few seconds of glitchy silence and then the interviewee continued to speak while the glitchiness continued for a few more seconds. I can't be 100% sure that its the glitchiness but then again, the word loops only happen there? Hope this helps! Like you suggested above, I just restarted the transcription from just after the word looping point and all is good. |
Beta Was this translation helpful? Give feedback.
-
|
I've made the same observation: If there is a silent part with some background noise, whisper may start to hallucinate text which can then result in a loop. Maybe a VAD filter would help to filter these silent parts out - something to test for the future. |
Beta Was this translation helpful? Give feedback.
-
|
I observed a very strange behaviour in the GUI that led to an increase in the speed of the embedding and segmentation process by approximately 50%. For some reason, when minimizing and maximizing the NoScibe window during the embedding and segmentation part, it appears to utilize more cores and result in a time reduction of roughly half. I tested this with a 15-minute audio file, comparing waiting versus minimizing and maximizing the NoScibe window. The results were striking: it took 6 minutes to wait for the process to complete vs. just 3 minutes when toggling the NoScibe window. Initially, I thought that Python's buffering might be slow and only load the UI when in focus, but the time difference tells a different story. |
Beta Was this translation helpful? Give feedback.
-
|
Did some tests on my AMD Ryzen AMD 7 6800U mobile processor, no cuda. I'm pretty sure this has to do with Windows 11 prioritizing foreground processes to 'improve user experience', see: https://superuser.com/questions/1763751/windows-10-11-cpu-performance-lowers-to-half-when-computational-task-is-not-in-f |
Beta Was this translation helpful? Give feedback.
-
|
In my case, to gain full speed the Windows Taskmanager or an other app (tested with Firefox) has to stay open and on focus. Full Speed: noScribe Window open, Taskmanager/Firefox window on focus |
Beta Was this translation helpful? Give feedback.
-
|
Forgot to mention: |
Beta Was this translation helpful? Give feedback.
-
|
It gets a bit stranger. Today, while using noScribe, I noticed that when the noScribe window is open and in focus during the faster whisper part, and when I move the mouse in a circular motion during the text generation part, it parses faster compared to when the noScribe window is open and in focus. So it's not just a matter of being in focus or out of focus, but also how the mouse interacts with the running software UI. I haven't tested it with a laptop-grade CPU yet. My tests were conducted on an AMD 3700x and an Intel 2699v3, both of which are desktop-grade CPUs. So, I don't think that it's related to the E-cores in Intel or the energy-efficient cores in AMD. |
Beta Was this translation helpful? Give feedback.
-
|
Here's how I tested it: It seems that interacting with the widget of the Tkinter UI causes the most significant increase in speed. I used PyAutoGUI to hover in a circular motion around the audio file select folder widgets, and it caused a huge increase in speed. mouse_widget_tk.mp4When running the script, please be aware that it locks the mouse in a circular motion within specific coordinates for 60 seconds. The only way to exit is by keeping the terminal open and pressing Ctrl+C to quit or using Ctrl+Shift+Del to force interruption. import pyautogui
import time
import math
# Display warning message
print("WARNING: This script will lock the mouse in circular motion in the 200 x 250 coordinates for 60 seconds.")
print("The only way to exit is by keeping this terminal open to press Ctrl+C to quit or Ctrl+Shift+Del to force interruption.")
print("Please ensure that you are running this script intentionally.")
# Define the center and radius of the circular motion
center_x = 200 # Update with the x-coordinate of the center of your window
center_y = 250 # Update with the y-coordinate of the center of your window
radius = 50 # Adjust the radius as needed
# Define the duration for which the script will run (in seconds)
duration = 60 # Adjust as needed
# Define the number of steps to complete the circular motion
num_steps = 15 # Adjust as needed
# Calculate the angle increment between each step
angle_increment = 2 * math.pi / num_steps
# Track the start time
start_time = time.time()
try:
while time.time() - start_time < duration:
for i in range(num_steps):
# Calculate the angle for this step
angle = i * angle_increment
# Calculate the coordinates of the point on the circle
x = center_x + radius * math.cos(angle)
y = center_y + radius * math.sin(angle)
# Move the mouse cursor to the calculated coordinates
pyautogui.moveTo(x, y)
# Wait for a very short duration before the next step
time.sleep(0.001) # Adjust as needed for faster motion
except KeyboardInterrupt:
# Stop the script if Ctrl+C is pressed
pass |
Beta Was this translation helpful? Give feedback.
-
|
Hello A new version of noScribe (0.6) is ready with a lot of improvements, but we are still ironing out some issues with Mac. I can't test it myself since I don't have NVIDIA graphics in my system. What irritates me a little is the fact that the compiled version with cuda is now about the same size as the non cuda version. I hope that everything is included correctly. The new version should be much faster overall. On CPU, I see a 3 to 5 times speed improvement in the transcription phase. I'm curious if this is the same with cuda. |
Beta Was this translation helpful? Give feedback.
-
|
A new version is available under the same link: https://drive.switch.ch/index.php/s/HtKDKYRZRNaYBeI |
Beta Was this translation helpful? Give feedback.
-
|
Awesome! No Mouse-Moving or other special tricks anymore! On Nvidia RTX 4090 ~1h Audio pyannoted & transcribed in <12 minutes... And working at the same time on something else... THANKS! |
Beta Was this translation helpful? Give feedback.
-
|
@paskima: Thanks, sounds nice! |
Beta Was this translation helpful? Give feedback.
-
|
@kaixxx I've tried the new V0.6 build again. The "Torch error" is fixed but no speed improvement. V0.6 on focus: 155 Minutes inputfile: Die Schatzinsel (342 min)
|
Beta Was this translation helpful? Give feedback.
-
|
@JB-FHE It seems as if v 0.6 is not using CUDA at all in your case. |
Beta Was this translation helpful? Give feedback.
-
I've compiled a new version that should support acceleration via NVIDIA graphics cards using cuda. It's not officially released yet. Could you test and report back if it works? Thanks!
Installer here: https://drive.switch.ch/index.php/s/EIVup04qkSHb54j?path=%2FnoScribe%20vers.%200.4.5%2FWindows%2Fcuda%20version
EDIT (09.04.2024): It appears to function quite well. Here are some tips gathered from the discussion below:
Beta Was this translation helpful? Give feedback.
All reactions