Deep Reinforcement Learning

Holy shit, this is amazing. I've been playing with OpenAI's Gym environment to write bots based on Deep Reinforcement Learning and I've impressed by how they've made it so easy to use despite not knowing much about AI. Makes me wonder what you can really do if you learn the subject more thoroughly.

Anyway, this got me thinking about the possibility about writing bots for more complex games. I looked into FIFA 18 but there are no indicators on the screen showing which team is in possession of the ball so I'm not sure what kind of immediate reward I can assign to an action. There's also no visual indication whether it's still in the game or changed to a cut scene so this seems to require a lot of manual work before I can get a bot to learn.

Should I try some other game instead? Any thoughts about the best game titles that may be suitable for bots?

Attached: 1545224343291.jpg (500x488, 23K)

Other urls found in this thread:

arxiv.org/pdf/1704.00710.pdf)
twitter.com/wisdomofchopra?lang=en
developer.valvesoftware.com/wiki/Dota_Bot_Scripting
youtube.com/watch?v=rvnHikUJ9T0
github.com/mystic123/tensorflow-yolo-v3
youtube.com/watch?v=zxFHU9DhR4Y
twitter.com/SFWRedditImages

Fighting games

Make a bot play DiveKick and work from there, as you go towards classic games like Street Fighter 2, Tekken 3, etc and up towards stuff like SF5 and Tekken 7 the requirements of the bot will get more complex as the games get faster and add more competitive mechanics

Attached: GrandioseCreepyHapuku-max-1mb.gif (260x146, 889K)

I have never played FIFA and I dont enjoy soccer in general, so this is a guess. I think FIFA mostly about motoric-skills/reaction-time, than about strategy. So I it should be an easier candidate for writing a deep-learning bot than any RTS or board games.

Your general concept isnt quite right though. You should never use the graphics-output of game as input. As you found out yourself, the graphical state does not represent the full game state in a single frame, unless you're playing pacman or alien-invaders. And theres way too much noise (useless information, eye candy).

The trick is to debug the program (FIFA) and find out where the information you need (game state, like possession of playball etc) are stored in memory and how to interpret them. With this you have much better control over what you're using to train your AI.

Well, this is true but isn't feeding a sequence of frames to an LSTM figure out what information is useless?

For instance, I've seen a self-driving implementation for Need For Speed and it was only using pixels as the input and it used YoloFlow to detect obstacles and it learned to drive within a few weeks.

I'm only beginning to work with AI, so you probably know more about that than I do. Filtering the noise surely is one application of an AI, but I think of it more as extra feature. Your real goal isnt filtering information though - your real goal is to know the best possible move at any given turn / tick. (Which translates to: your goal is not to filter the noise, but to win the game)

Complex games require tonnes of GPU power for training an AI. The Dota2 AI bot was trained using 256 GPUs.

Hey, I'm also completely new to this. You're right in that having objective information about the game's state will make the bot more efficient at achieving the goal.

There are two types of environments in the OpenAI universe. Some are RAM based, and some are just pixels from the VNC channel of a virtual machine but those games are rather simple games. There used to be a GTA V environment but they seem to have removed it due to licensing issues.

This paper (Link: arxiv.org/pdf/1704.00710.pdf) for example, describes a process to generate 3D models from 2D images which might be useful for a game bot to use as one of the inputs in addition to pixels.

Picture unrelated.

Attached: demo.gif (600x243, 906K)

Meant to reply to

Can I use this to make an AI that observes a specific subset of Twitter accounts and tries to imitate/simulate that type of person? Are there any guides for a brainlet with no AI experience to experiment with something like this?

There are Twitter and Reddit datasets for download. Start by lumping together users who have a similar style of tweeting and then use chatbot-rnn (GitHub) to train it.

twitter.com/wisdomofchopra?lang=en

Considering what this guy said (I dont play dota)
>The Dota2 AI bot was trained using 256 GPUs.
I bet they weren't using video input, it seems like an officially supported project. And as far as I can tell, the dota AI is not on the same level as the best human players. Unlike in Go, Chess, etc.

So it's basically up to your intention of why you are writing the bot. If it's for learning-AI purposes, throwing away time and/or money for computation power just to filter out the noise received on the video input doesn't seem reasonable.

If you're into this for making money by selling/running bots on profitable online games (theres a whole blackmarket industry for this), then I guess neither FIFA nor using video-input is the way to go. Choose a more profitable game and read from the game's memory to save time/power/money.

If you're concerned about the legal aspect, debugging for learning-purposes is non-punishable in virtually every country (excluding america, probably). Just dont sell your bot and dont use it online in this case.

If your game of choice is only available on the current-gen consoles but not on PC, then it's probably video-input. But you'll need a video-capturecard for that I think.

Also, if you don't know anything about debugging / software development / computer memory, and you're not into game-AI-development for the long run, video-input will probably yield faster results. For the long run I think it'll be worth learning the debugging skills required.

They used the API at developer.valvesoftware.com/wiki/Dota_Bot_Scripting

I'm not looking to sell anything nor worried about any legal issues. I'm mainly interested in exploring what can be done with computer vision and games are a perfect virtual world to experiment and I'm not trying to beat any game so it doesn't have to be perfect. AI is new to me so I'm just curious to see whether it will just perform random actions or be able to learn anything at all. I do have interest in consoles as well and planned to use a Magewell HDMI capture device for the video information.

It's not that I don't know how to debug games but my main goal here is to be able to teach it how to play using the same information that is available to a human player, so I would consider looking at debug information and memory dumps to be kind of a way to cheat (no pun intended).

Attached: detect.jpg (1280x720, 138K)

youtube.com/watch?v=rvnHikUJ9T0

As I said i'm just getting into deep learning and image recognition. I haven't looked into openAI at all yet. I've just begun a project that is supposed to be used in a production-line of some very small items (1x1x1 cm). I'm using a camera to detect the presence, absence or partial presence (broken into pieces) of said small items. I've just finished the electronics and software part (taking the image at the right time, camera parameters etc) so this week I'll start collecting images for the learning set. I'm reading through the openCV docs for that atm.

github.com/mystic123/tensorflow-yolo-v3

This works very well for that purpose. You just take a video of the objects you want it to detect and separate them into frames and train it. Then this can detect them in new images easily.

forget to attach the image. in case anyones interested, this is what im trying to differentiate.

Attached: goodbad.png (284x263, 62K)

Speaking of Tekken
youtube.com/watch?v=zxFHU9DhR4Y

Attached: 1545646127822.jpg (752x1080, 159K)

If you can send me a couple of hundred images of good and bad, I can take a crack at it.

well if u're interested I suppose I could, once I have them myself. As I said i just finished the setup. And im also considering to ditch the 2MP camera for 4MP.

Some things you'll need to know in advance: I need to check one image every 1 to 2 seconds (speed is dynamic).
Every image I need to check consists of 30 of those holes/pills, in sets of 10. If one of a set is bad, the whole set (10 pieces) are trashed.
And ofcourse, this is a real-time application, so the result for each set in every image must be ready before the next image is taken.

ps:
if you're still not feeling challanged: all of this is supposed to run on an ARM SoM, Mali400 GPU and 4x 1.2GHz ARM-A8. If possible.

2 MP should be more than enough for the classification task.

Is there any limitation on the hardware this needs to run on?

see However this isnt a fun project. The resources will ultimately be scaled to match the requirements, even if that means buying 4x 1080ti.
However I'd personally be very happy to keep it on an ARM module. The signal when to take the image isnt the only other interface; I also need to control a few other things via GPIOs, I2C and PWM. Which means if I have to use an x86 machine, I will also need a second device/board to do the GPIO stuff. Which means tedious and unrewarding work.

>however
Im also stoned since I just got home from work. excuse my poor language.

Would a fast paced racing game like TrackMania be suitable for this? It seems like a fascinating project if I could make it work somehow. I'm pretty sure it's easily scriptable via Openplanet, too.