What is ReliableGPT?
reliableGPT is your go-to solution for ensuring 100% uptime and reliability when using OpenAI's language models. Whether you're dealing with rate limits, timeouts, API errors, or key issues, reliableGPT has got your back. It’s designed to keep your LLM (Large Language Model) applications running smoothly, no matter what.
What are the features of ReliableGPT?
- Model Fallback: Automatically retries failed requests with alternate models like GPT-4, GPT-3.5, and more.
- Caching: Stores responses for future use, ensuring users get a reply even during high traffic or server overload.
- Key Rotation: Handles invalid API key errors by rotating through backup keys.
- Context Window Management: Retries requests with models that have larger context windows when needed.
- Custom Fallback Strategies: Define your own fallback model sequence for specific error handling.
What are the use cases of ReliableGPT?
- High-Traffic Applications: Ensure all users get a response, even during peak times.
- Azure OpenAI Users: Seamlessly switch to raw OpenAI endpoints if Azure fails.
- Key Rotation Scenarios: Automatically handle rotated or invalid API keys without downtime.
- Context Window Errors: Retry requests with models that can handle larger inputs.
How to use ReliableGPT?
- Install the Package: Run
pip install reliableGPTto get started. - Integrate with OpenAI: Replace
openai.ChatCompletion.createwithreliableGPT(openai.ChatCompletion.create, user_email='your_email@example.com'). - Set Fallback Models: Optionally define a fallback strategy like
['gpt-3.5-turbo', 'gpt-4']. - Enable Caching: Turn on caching with
caching=Trueto handle overloaded servers. - Add Backup Keys: Use
add_keys()to store and rotate through multiple API keys.
