Sunday, February 21, 2016

WidapMind update

So close and yet, so far.

First of all, if you think the above program is simply parroting back what I tell it, take another look. You will see that it is changing the forms of the words and the structure of its replies are independent of how I gave it the info. This can only be accomplished by interpreting the meaning behind the actual words.

I've been working on a natural language processor/AI in Java for a week or two now. Yes, I'm making an AI and I'm doing it in Java. If you know me well, you'll know that the latter is the more surprising.

I chose Java over my usual C++ because my primary target for this project is an Android app. I've even written a primitive Android front-end for it but I'm currently putting most of the work into the back-end, which I run as a console Java program. In developing this, I was surprised to find that Java isn't a completely useless language. It's String class is quite nice and, as much a I hate to admit it, garbage collection does have its advantages.

So far, there are four main components to WidapMind. The before mentioned front-end, the input text parser and interpreter, the dictionary and the all important database of things. All of these (except the Android interface) work together to produce the conversation below.

I load and parse a text-format dictionary, which I use to determine the parts of speech of words and to switch between different versions of the same root word (though I can do a decent job of that automatically if the word isn't an exception to any rules).

The text parser is the component that needs the most work. It looks like its doing a good job here (except for the very end) but it needs a complete overhaul. It can currently only interpret very specific sentence structures that I have explicitly programmed in. That's fine for demos but it will be next to useless in any real application and its completely non scalable. What I need is to automatically break the sentences down into a structure tree that can be parsed regardless of the sentence complexity, but this will be a big job. The data structuring alone will be a mess unless I get everything perfect.

The database component is fairly straightforward. The parser figures out that the user is saying a particular thing has a particular property, and so the database stores the information. If the parser later requests some info about something, the database does a search and returns it. How it is formatted gets a bit tricky internally but the concept is simple.