I want to setup a livestream using the RTMP and NGINX server for a DJI drone.

[ShabanWaris1234]

I want to set up a live stream using the RTMP and NGINX server for a DJI drone. I have already trained a custom yolov8 model. However, I am having a problem setting up a live stream with the RTMP and NGINX server for my drone. Are there any tutorials or codes that can guide me in setting up a real-time detection using a drone? I also need to make a web app for my project I plan to use flask. Any tutorials would help.

[glenn-jocher]

Certainly! Setting up a live stream with RTMP and NGINX for your DJI drone and integrating it with your custom YOLOv8 model for real-time detection sounds like an exciting project! Here’s a brief guide to get you started:

1. RTMP Server Setup with NGINX:

   • Install NGINX and the RTMP module. You can find a good tutorial on how to do this here.
   • Configure NGINX to receive the RTMP stream from your drone. You'll need to modify the nginx.conf file to include the RTMP module settings.

2. Integrate YOLOv8 for Real-Time Detection:

   • Use the Python cv2 library to capture video frames from the RTMP stream.
   • Load your trained YOLOv8 model and pass the video frames to it for detection.

   import cv2
   from ultralytics import YOLO
   
   # Load model
   model = YOLO('path/to/your/custom_model.pt')
   
   # Capture RTMP stream
   cap = cv2.VideoCapture('rtmp://localhost/live/stream_name')
   while cap.isOpened():
       ret, frame = cap.read()
       if ret:
           # Make predictions
           results = model(frame)
           results.show()  # Display the frame with detections
       if cv2.waitKey(1) & 0xFF == ord('q'):
           break
   cap.release()
   cv2.destroyAllWindows()

3. Developing a Web App with Flask:

   • Flask is a great choice for setting up a web server. You can start by creating a basic Flask app that displays the video stream.
   • Integrate the Flask app with the detection model to display the processed frames.
   • Check out the Flask documentation for basic setup and this tutorial for streaming video through Flask.

These steps should help you set up your project. For specific issues or errors, feel free to ask for more detailed help! Good luck with your real-time detection system! 🚀

[mordka]

@ShabanWaris1234 just a note, you won't get realtime detection, because RTMP has got few seconds latency embedded in protocol design. I've built realtime AI drone streaming pipeline with <1s AI latency and I'm open to share the details. If you are interested in this reach out to me on [email protected]

[kylegraupe]

Hi all,

It looks like I was in fact able to find a solution to this problem. You can check it out on this github repo. It took me some time to get it fully functioning, but I have posted the open-source code for everyone to see. If you would like to contribute, please do! There are application trailers on my website that show the application in action. If you see any issues, please add an issue to the board or fork the repo!

@glenn-jocher @ShabanWaris1234 @mordka

[glenn-jocher]

Thank you for sharing your solution and open-sourcing the code! This is a great contribution to the community. For any questions or feedback, users can engage directly on your GitHub repo. Keep up the great work!

[kylegraupe]

Thank you!

Please leave a star or a follow if you find it helpful! It means a lot for the code to be discovered by others looking to contribute. @glenn-jocher

[kylegraupe]

If you'd like to learn more, please visit my website here where you can see recordings from various test flights. If you have any specific questions regarding the use of the application, have any requests, or would like to work on a project together, my inbox is open!

Below are a few screenshots of a from test flights, where I have a segmentation mask next to the original frames in 720p (actually 704x1280 due to U-Net resizing necessity).

P.S. - I am currently looking for employment as a ML Engineer or Data Scientist. I am pursuing my second Master's degree, M.S. in Machine Learning, and already hold two degrees: M.S. in Modeling and Simulation Engineering and a Bachelor's in Aerospace and Ocean Engineering. Should this project impress, please let me know if there is a good fit!

@glenn-jocher @mordka @ShabanWaris1234 @ccamacho

[glenn-jocher]

Thank you for sharing your work and resources with the community! Unfortunately, we’re unable to assist with private collaboration or employment requests, but your project is impressive, and we encourage others to explore your repository and website for more details. Best of luck with your endeavors!