1.1.2

What's Changed

• feat: e2e test streaming by @justinmerrell in #93
• Update CI-e2e.yml by @justinmerrell in #94
• Add get_pods() API method by @alexeevit in #95
• API Improvements With Error Handling by @justinmerrell in #96
• Connectionpool fix by @justinmerrell in #97

New Contributors

• @alexeevit made their first contribution in #95

Full Changelog: 1.1.1...1.1.2

1.1.1

What's Changed

• Preserve the final job result's post condition in streaming scenario. by @Jorghi12 in #88
• Bump fastapi[all] from 0.101.0 to 0.101.1 by @dependabot in #91
• Create CI-e2e.yml by @justinmerrell in #87
• Update README.md by @justinmerrell in #92

Full Changelog: 1.1.0...1.1.1

1.1.0

Bug Fix

Fixes bug where our ping health monitor would stop if the handler received a blocking function.

What's Changed

• Bump fastapi[all] from 0.100.1 to 0.101.0 by @dependabot in #85
• Heartbeat thread by @justinmerrell in #86

Full Changelog: 1.0.1...1.1.0

1.0.1

What's Changed

• Data Race in TakeJob by @Jorghi12 in #84

Full Changelog: 1.0.0...1.0.1

1.0.0

We're thrilled to announce the release of RunPod 1.0.0! After refining our CI/CD pipeline and gathering valuable feedback from thousands of users, we are ready to introduce significant enhancements and fixes.

New Features

Multi-Job Concurrency

Workers are now smarter and more efficient. They can fetch and process multiple jobs in parallel, accelerating your workflows and productivity. Here's what's new:

• Flexible job fetching: You can now fine-tune a worker's operations. When starting a worker, pass a function into the concurrency_controller. This function determines the number of jobs a worker should fetch in parallel.

Job Streaming

Our platform offers a powerful streaming feature that allows users to receive real-time updates on job outputs. This is particularly useful when dealing with Language Model tasks. We support two types of streaming generator functions: regular generator and async generator.

Regular Generator Function:

def generator_streaming(job):
    for i in range(5):
        output = f"Generated token output {i}"
        yield output

Async Generator Function:

async def async_generator_streaming(job):
    for i in range(5):
        output = f"Generated async token output {i}"
        yield output
        await asyncio.sleep(1)  # Simulate an asynchronous task (e.g., LLM processing time).

Usage:

To enable streaming, use either a regular or async generator function to yield the output results. The generator will continuously produce token outputs, streamed to the client in real-time.

How to Stream:

To utilize the generator function for streaming, you must use the /stream/{jobid} endpoint. Clients will receive the streaming token outputs by making an HTTP request to this endpoint. The jobid parameter in the endpoint URL helps the server identify the specific job for which the streaming is requested.

Real-Time Updates:

Once the streaming is initiated, the /stream/{jobid} endpoint will continuously receive generated token outputs as the generator function yields them. This provides real-time updates to the client or user, ensuring they can access the latest results throughout the job execution.

Using generator-type handlers for streaming, our platform enhances the user experience by delivering dynamic and up-to-date information for tasks involving Language Models and beyond.

Updates & Bug Fixes

We've also made some crucial improvements and squashed a few bugs:

• Improved Initialization: The worker implementation now leverages asyncio, significantly improving initialization times. Your workers are now ready to go faster than ever!
• Cohesive File Naming: We've renamed some files to improve cohesion and understanding. Now it's even easier to understand the purpose of each file in the project.

What's Changed

• build(deps): bump actions/setup-python from 3 to 4 by @dependabot in #46
• Add code coverage by @justinmerrell in #50
• bringing branch up to date by @justinmerrell in #51
• Api wrapper tests by @justinmerrell in #52
• Update refactor branch by @justinmerrell in #53
• Unit tests by @justinmerrell in #54
• Update setup.cfg by @justinmerrell in #55
• 48 heartbeat refactor by @justinmerrell in #49
• More tests by @justinmerrell in #60
• added documentation for Endpoint and create_pod by @therealadityashankar in #61
• Enabling multi-job, single-worker in Runpod-Python. by @Jorghi12 in #63
• Update init.py by @justinmerrell in #65
• rebase by @justinmerrell in #67
• minor worker refactoring by @justinmerrell in #66
• Worker loop cleanup by @Jorghi12 in #68
• status_code > status by @justinmerrell in #69
• Update rp_ping.py by @justinmerrell in #70
• Fixing "timeout context manager should be used inside a task" by @Jorghi12 in #71
• Cleanup by @justinmerrell in #72
• Bump fastapi[all] from 0.99.1 to 0.100.1 by @dependabot in #73
• Bump nest-asyncio from 1.5.6 to 1.5.7 by @dependabot in #74
• Update rp_scale.py by @justinmerrell in #75
• Polish by @justinmerrell in #76

New Contributors

• @dependabot made their first contribution in #46
• @therealadityashankar made their first contribution in #61
• @Jorghi12 made their first contribution in #63

Full Changelog: 0.10.0...1.0.0

0.10.0

New Features

Test API Server

• We introduced the ability to quickly deploy a locally hosted API server with your worker code. This is accomplished by calling your handler file with the --rp_api_serve argument. This allows for faster, more flexible testing environments. Check out our blog post for an example.

Log Level

• For better control over logging, we now allow setting the log level by calling your handler with the --rp_log_level set to the desired level. If this argument is present, it will override the RUNPOD_DEBUG_LEVEL.

Updates

Logger

• We've made significant improvements to our logger for better clarity and control:
  • logger.py has been refactored and renamed to rp_logger.py for consistency.
  • Logging level is defaulted to DEBUG by default. Note that when your worker runs on RunPod we set it to ERROR unless otherwise set.
  • Breaking Change: RUNPOD_DEBUG no longer controls whether or not logs are printed. To prevent all logs from printing, set RUNPOD_DEBUG_LEVEL to 0 or call your handler file with the argument --rp_log_level="NOTSET"

RunPod API Python Language Library

• We've added more options for creating Pods:
  • You can now specify data_center_id and country_code when creating Pods, allowing for more precise control over pod creation.

What's Changed

• feat: pass data_center_id to create_pod by @shibanovp in #40
• feat: add country_code to create_pod by @shibanovp in #42
• Logger Update by @justinmerrell in #43
• updating branch by @justinmerrell in #44
• API Server For Local Testing by @justinmerrell in #45

New Contributors

• @shibanovp made their first contribution in #40

Full Changelog: 0.9.12...0.10.0

0.9.12

Updates

Error Handling

• Job returns are filtered to match the type dictionary before looking for the error key. This will prevent false positives from matching the term "error" in a returned body.
• Exceptions raised by the handler now include additional information, including the hostname and pod id. The formatting has also been improved.
• An explicit exception regarding missing or invalid RunPod API key is now raised when making endpoint calls.

What's Changed

• Handle 401 error for serverless calls by @arsenyinfo in #38
• Add informative error message when API key not provided by @LukeWood in #39

New Contributors

• @arsenyinfo made their first contribution in #38
• @LukeWood made their first contribution in #39

Full Changelog: 0.9.11...0.9.12

0.9.11

New Features

Job Streaming with Generator

• If your handler contains yield it will be treated as a generator and can stream the results.

Updates

• GitHub action requirement version updates.

Refactors

Logging

• Altered many log levels from info to debug. This is in preparation for RunPod persistent logging release.
• Reformatted logs to help identify job associations.

Bug Fix

• Checks if the handler result is a dict before looking for error and refresh_worker keys

What's Changed

• More Informative Errors by @lapp0 in #26
• Feature/new api by @SkullMag in #30
• updating branch by @justinmerrell in #32
• Version updates by @justinmerrell in #33
• Streaming job support with generators by @winglian in #28
• add ExtraArgs support for upload_file_to_bucket by @slep0v in #29

New Contributors

• @lapp0 made their first contribution in #26
• @winglian made their first contribution in #28
• @slep0v made their first contribution in #29

Full Changelog: 0.9.10...0.9.11

0.9.10

New Features

Debugger Utility

• We've introduced a new Debugger Utility that allows you to set timing checkpoints in your code. This functionality can help you identify bottlenecks and optimize your application more effectively. Debugger Docs

Improvements

rp_downloader

• Performance enhancement: rp_downloader now supports streaming downloads with chunking. This update results in faster, more efficient file downloads.

Dependency Updates

• We have updated the versions of various dependencies to ensure optimal performance and compatibility. Please take a look at the updated requirements file for the specific changes.

Bug Fixes

Heartbeat Thread Management

• We have addressed an issue concerning the management of heartbeat or ping threads. The fix improves the stability of the module and reduces the likelihood of encountering threading errors.

Full Changelog: 0.9.9...0.9.10

0.9.9

Fix:

• Timed out Jobs were not being handled gracefully

Full Changelog: 0.9.8...0.9.9