openai-manager | OpenAI requests by balancing prompts

Install openai-manager on PyPI. Prepare your OpenAI credentials in.
Install openai-manager on PyPI. pip install openai-manager
Prepare your OpenAI credentials in Environmental Varibles: any envvars beginning with OPENAI_API_KEY will be used to initialized the manager. Best practice to load your api keys is to prepare a .env file like: OPENAI_API_KEY_1=sk-Nxo****** OPENAI_API_KEY_2=sk-TG2****** OPENAI_API_KEY_3=sk-Kpt****** # You can set a global proxy for all api_keys OPENAI_API_PROXY=http://127.0.0.1:7890 # You can also append proxy to each api_key. # Make sure the indices match. OPENAI_API_PROXY_1=http://127.0.0.1:7890 OPENAI_API_PROXY_2=http://127.0.0.1:7890 OPENAI_API_PROXY_3=http://127.0.0.1:7890 Then load your environmental varibles before running any scripts: export $(grep -v '^#' .env | xargs) YAML config file: you can add more fine-grained restrictions on each API key if you know the ratelimit for each key in advance. See example_config.yml for details. import openai_manager openai_manager.append_auth_from_config(config_path='example_config.yml')
Run this minimal running example to see how to boost your OpenAI completions. (more interfaces coming!) import openai as official_openai import openai_manager @timeit def test_official_separate(): for i in range(10): prompt = "Once upon a time, " response = official_openai.Completion.create( model="code-davinci-002", prompt=prompt, max_tokens=20, ) print("Answer {}: {}".format(i, response["choices"][0]["text"])) @timeit def test_manager(): prompt = "Once upon a time, " prompts = [prompt] * 10 responses = openai_manager.Completion.create( model="code-davinci-002", prompt=prompts, max_tokens=20, ) assert len(responses) == 10 for i, response in enumerate(responses): print("Answer {}: {}".format(i, response["choices"][0]["text"]))