OpenAI Vision Demo
A downloadable demo for Windows
Demo demonstrates the capabilities of the Complete OpenAI API plugin:
A user takes a screenshot of the game. Then the image from the camera will be sent to OpenAI for object and context recognition. OpenAI will return a description of what is shown in the image. Then this description is sent again to OpenAI for speech generation.
Controls
ENTER - take a screenshot and send it for processing
SPACE - listen to audio again
Download
Install instructions
You will need to create an key for OpenAI requests to run the demo:
The next step is to add the generated key to the following file VisionDemo/OpenAIAuth.ini
APIKey=sk-xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx OrganizationID=org-xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
OrganizationID can be left empty.
Comments
Log in with itch.io to leave a comment.
This looks promising. I added the API key, the demo starts, I can move around in the space and take screenshots which seem to get processed, but there's no audio. Any idea why?
Thank you for reporting this. There was an issue with incorrect path for audio in packaged project. I have updated the binaries.
I downloaded the VisionDemo.zip again, but still no sound.. Should it show the text overlay of the spoken audio like in the screenshots? It's not doing that either. It only shows "Scanning the environment" for a few seconds and then nothing happens.
Yes it should. I've just tested on another PC and the current build works for me.
Should work like in this demo
But if there is no text it sounds like a problem with the OpenAI server connection.
There should also be an audio file received from OpenAI:
There are two more demos you can test to make sure your API key is valid:
https://life-exe.itch.io/openai-plugin-demo
https://life-exe.itch.io/openai-museum
I now tried the "OpenAI Museum" demo with a new API key on 2 different accounts and it shows in the app that I reached a usage limit, even though in the OpenAI accounts it only shows 1 GPT-3.5-turbo request is being made, no other usage. Does this demo require a paid OpenAI account to work?