**reliableGPT** is your go-to solution for ensuring 100% uptime and reliability when using OpenAI's language models. Whether you're dealing with rate limits, timeouts, API errors, or key issues, reliableGPT has got your back. It’s designed to keep your LLM (Large Language Model) applications running smoothly, no matter what.

What are the features of ReliableGPT?

* **Model Fallback**: Automatically retries failed requests with alternate models like GPT-4, GPT-3.5, and more. * **Caching**: Stores responses for future use, ensuring users get a reply even during high traffic or server overload. * **Key Rotation**: Handles invalid API key errors by rotating through backup keys. * **Context Window Management**: Retries requests with models that have larger context windows when needed. * **Custom Fallback Strategies**: Define your own fallback model sequence for specific error handling.

What are the use cases of ReliableGPT?

* **High-Traffic Applications**: Ensure all users get a response, even during peak times. * **Azure OpenAI Users**: Seamlessly switch to raw OpenAI endpoints if Azure fails. * **Key Rotation Scenarios**: Automatically handle rotated or invalid API keys without downtime. * **Context Window Errors**: Retry requests with models that can handle larger inputs.

How to use ReliableGPT?

1. **Install the Package**: Run `pip install reliableGPT` to get started. 2. **Integrate with OpenAI**: Replace `openai.ChatCompletion.create` with `reliableGPT(openai.ChatCompletion.create, user_email='your_email@example.com')`. 3. **Set Fallback Models**: Optionally define a fallback strategy like `['gpt-3.5-turbo', 'gpt-4']`. 4. **Enable Caching**: Turn on caching with `caching=True` to handle overloaded servers. 5. **Add Backup Keys**: Use `add_keys()` to store and rotate through multiple API keys.

ReliableGPT Contact Email:

ishaan@berri.ai, krrish@berri.ai

What happens if my API key gets rotated?

reliableGPT will automatically rotate through your backup keys to ensure zero downtime.

Can I define my own fallback models?

Yes, you can specify a custom fallback strategy like `['gpt-4', 'gpt-3.5-turbo']`.

How does caching work?

reliableGPT stores responses in a Supabase table and uses cosine similarity to retrieve similar cached responses when needed.

Can I use this with Azure OpenAI?

Absolutely! reliableGPT can fall back to raw OpenAI endpoints if your Azure OpenAI requests fail.

ReliableGPT

5.0

19.25%

reliableGPT ensures your LLM applications never fail, handling rate limits, timeouts, and key errors with ease.

Added on: 2025.03.08

Social Media:

#Startup Tools

Visit Site

Share This Tool: