Automated Web Surfing Software: Automethod Web Suite

I was assigned to a very simple mission some time ago:

"We would like to be alarmed if customers are not able to proceed to the order checkout page."

Yes, we can periodically run a script and use curl in it to check if the http requests are successful, and make it send email notifications to the developers.

But hey, why don't we do some overkillin' after that and write an automated web agent just for fun? I want a real user surfing on the web by clicking, scrolling and such. Controlling http requests is lame anyways.

Maybe something like this:

As I always do, I'm going to give you a reason to continue reading: That guy can play any browser based game by its own! 

Before moving to the "blah blah section", check this video out: http://youtu.be/po9_ERgTics It demonstrates how to use the program for a very simple conditional routine:
- Open some page,
- If it is google and there is a search field on the page, search for "hello" and select the first link,
- Otherwise go to another page.
- Do this in a loop, with 5 seconds delay

 (Unfortunatelly the video is in Turkish for now. English demo and tutorial is going to be uploaded with the last release.)

I used to be a fan of browser based games like you, then I didn't take an arrow in the knee and I'm still a huge fan. So why don't I have something which plays travian (or any other game) by itself? When I was playing ogame like hell, I would be very happy if I got an email like this: "Commander! <someguy> was attacking our fleet in <someplanet>! But don't worry, while you were sleeping in your comfortable bed with drool on your jaw, we managed to put all our resources to carriers and escaped the fleet just before the enemy arrived. They returned safely after he was gone. We also waited until the last second before escaping, just to screw with him."

Sending a message of this kind is just a funny glimpse of what this program can do. We can define variables, functions and routines on a GUI without writing any code. It allows the user to inject javascript on a page, by selecting pre-defined functions from drop down lists, or by writing javascript if desired.

What else can we do over the web with such a program? I'm not going to give examples here, use your imagination ;)

So what do we need for a program like that?

Let's start with this: I want to be informed when someone attacks me in ogame. How do I know if I'm being attacked, a red triangular sign appears on top of the page (very creative). So we simply need to identify the red sign, and take corresponding actions.

We need a "cause and effect" system

- If something occurs on the page, do something.

We need to define conditions and actions

conditions:
- something is present on the page.
- something is not present on the page.
- page url is something
- ... boolean result of any javascript or jquery which is executed on the page

actions:
- send email
- milk the cow
- beat the trash
- tickle the hair drier
- ... or any javascript/jquery to be run on the page if a condition is satisfied or not satisfied

We should be able to give meanings to html elements

- Red triangular sign => <img id='under_attack' src='you_are_screwed.jpg' ...>
- use firebug or something to manually get the id's and classes of the elements
- use XPATH to locate them easily, without being obligated to parse the DOM with your bare hands.

Ok, now I want my agent to click somewhere if something is present on the page. Clicking can be done in many ways; we can always use curl for get-post request and responses. The thing is; the web is MUCH MOAR than that. Especially while playing browser games, we are in a wild jungle of javascript, triggering events and making ajax calls like there is no tomorrow.


I can feel it now:
- assign the xpath of the red triangular sign which goes something like: //img[@class="under_attack"] to a variable
- do the same for the buttons to press, textfields to type, and dropdown menus to select
- define the condition: the red triangle is on the page
- define the action: make a series of clicks and selections on several places (each identified with their xpath's as variables) 
- send notification with pre-defined message template filled with variables.

- do all these, or some of them conditionally, on a routine.

If I use Selenium's browser primitives, I can focus on my main objectives more effectively. They already handle event triggering (note that they all use xpath's of the html elements to locate them) on a variety of browsers. Struggling with browser compatibility is not going to help me right now. I'm going to use selenium PHP exporter to get php binding methods of the following:

click, type, select (from dropdown) find (check if an element is present on the page)...

I'm going to wrap these primitives with additional functionalities and give the user the ability to use them as pre-defined functions for defining routines and scenarios, which are going to be used by the user again to create even more complex functions.

The program is completed some time ago, but I don't have time to release it in a convinient way. A demo video is available in Turkish btw.

Evolution of Lentus Online

Evolution of Lentus Online: A 3D MMORPG with XNA 4.0

You can watch the demo video of b1.2 here.

After a couple months of coding the game core, it became impossible to test the functionality with a text based client. Yes, I used a text based client in v1 to test the game, when it only included moving around, acquiring items, attacking people, dying and looting etc.

Although I'm a C++ and OpenGL type of guy, I preferred new XNA 4.0 (a very good DirectX wrapper) for graphics and C# for coding the server and the client; surrendering to insistent suggestions of a friend. Honestly, I was too lazy at that time to dig into low level stuff (like in my other simulation projects), since I was just starting to deal with two majors in the university.

Behold... The ultra mega killer kamehameha client:

I made use of a project on the internet, whish I could remember which one, but if you type "XNA for dummies" or sth like that on google, you would eventually see that project.

Yeah I know, shame on me.











Before moving to the other steps, I want to show you some scenes from v5, to impress you and give you a reason to read the rest:

Me and my friends, capturing a castle, which the bots inside gathered together and built by themselves.

Sorry for using the same free models for everything, but I was a student, you know, a poor guy in his dorm eating bread with water.

*dramatic violin melody here*







As you may have noticed, I made use of Riemer's XNA tutorial for the terrain (who doesn't! You should go visit his great xna tutorial sometime), and the skinned animation project from apphub. I'm such a thieve right? Believe me, it takes much effort for even adopting these stuff into your project. And also note that XNA is not a game engine where you drag-drop items to the scene. It is a DirectX framework where you actually code stuff. They are two different things, people!

Here is another one from the same v5 beta release:

Note that these ships carry players & bots and can fire rocks if you load catapults in them. I don't know what you think, but it is pretty cool for me.

If you are curious about why I have that many versions, I mean why the fork do I have v5, it is just for me to not to confuse myself in over 40k lines of code when I need something from earlier versions. I suggest you to have a good versioning system of your own (unless you use plug-ins or other software for that) to make your life easier.

Also you can be satisfied by seing that the project is moving forward :)








Anyways, let's continue to our journey shall we? *like a sir*

The fundamental idea of the game: Everything is a container. There is a tree shaped object hierarchy. Everything has a parent container, and children containers. Even game effects are implemented as containers. For example, your right hand container has a sword, which is also carrying a cold effect. Yes, you can loose your hands during the battle, or someone can carry you around, and maybe even eat your leg and put it into his stomach container. I believed that this structure would give me a huge flexibility.

Here I managed to divide the screen for the game, map and inventory, and finally get the real time inventory working. This is actually the foundation of my container notification system.









Contrary to the common sense, containers don't know what they view. They in fact know the ones who are viewing them. If a container loses one of its children, it notifies the ones who are viewing it. Sounds strange, but this system proved its efficiency for me later in the developing.

Yes, that is the character panel, what were you thinking? I started to bind actions to the graphical interface starting around here I guess. I thought that clicking to an object would be a better way to attack instead of typing "canerus attacks zeus". Oh the pain...













Every action and item has an ID generated by the server, and these ID's should be interpreted by the client and the server separately. You don't have a character in this game, your client controls a container, and this container may change. For example, when you are driving a ship, you become the ship, and you can use actions that a ship can use.

Yes, it indeed looks like my grandmom's edgings, but... Have I said before that I suck interstellar-size at arts and drawing? Anyways, there are actually some important things beneath this picture.






Bots are able to track you, and intelligently use different tools against you. For example if you have a sharp and light knife, they take out a light shield and choose a weapon against your armor and shield. All body parts may have different armors, and bots determine your weak points according to their intelligence.

I'm using some new badass 3d models here. Also note the ultra-sucking tile map.













By the way, as the graphical client evolves at tortoise speed, huge amount of things are going on on the server side. Many new functionality were implemented and were waiting to be included in the client. If I remember correctly, by the time that this screen was taken, I was implementing the farm system and using tools to harvest faster etc. on the server. Note that the server runs all the game logic, where the client is only responsible for sending action requests and displaying responses accordingly.

As the client have slightly improved view, the game logic has minion and lineage system now. Containers which are in your lineage aid you in the game.










Instead of individually selecting the minion and giving an order, I implemented a loyalty system, in which your minions around you gets more loyal to you as you fight together, and you loose their loyalty if someone beats you up in front of them. According to their loyalty level, minions may ruthlessly attack your enemies till death, may flee and leave you like a tree, or just stand there and watch (traitor bastards).

You may have your own minions by zombifying corpses, have children by mating with other players, or paying vendors for mercenaries. Bots can have children too, they create their own lineage and hang around in groups.

Action panel is added and the character panel is improved. Actions can be used by mouse clicks now. Judging by the poison action, I can say that I was implementing timed effects at that time.










Effects are bonded to effect containers, and affect their parent containers. For example, poison event is bounded to a poison container, which is contained in your sword. When you hit somebody with that sword, the poison container is copied to them, damaging the body container over time. This is actually a very cool system, and I believe that it provides a huge set of opportunities for the game.

Constant effect: For example, the enemies agility drops down for a time period (according to the power of your spell) with the frost spell, and get back to normal again.


Periodic effect: Regeneration effects can be good examples to this type. If a ring has the health regeneration effect in it, it causes a periodic increase on its owners health. 

Effect stacking: e.g., frost spell may loose its effect on a container if it gets a fireball afterwards. lightning hurts more if you are frozen.

Just a couple pages of game menu is implemented on the client side, but the server had a lot more to do again!













Save and load systems are implemented. I started to optimize the server as the game logic started to become a real deal. The server manages its computing power over different parts of the map, making areas that no players are viewing sleep or work slowly. Many new technologies are implemented on the server side which I don't remember all of them now. For example the server sends the environment around a player when it has excess bandwidth and cpu, and the client caches these areas for further use.

Notification system is improved. For example when a player moves forward, new tiles are added to his field of view by adding him to their observers, and if something changes on these tiles, the player is notified. Also when the player changes tile, the new tile notifies its observers, and the observers respond to this notification according to the their own implementation. An aggressive bot may start chasing you as a notification response, as a trap may get activated and punch you in the nipples.

Clan system: Creature spawn is provided by mating. Bots reproduce and create lineages and clans.











If a clan becomes strong enough, they locate other clans around and attack them. I'm still having difficulties to keep the population in balance. Eventually a clan was screwing all their enemies and was dominating the map. I tried to prevent this with civil war;  if a clan becomes too strong, it is divided and forced to become enemies.

Age system: Creatures in the game gets older by time, and an older creature is stronger than the young ones. You may encounter ancient creatures at untouched parts of the map, and may not be able to see the sunshine again.

Wander system: Bots don't stand like trees, sometimes they move around randomly if they are idle. Usually they are busy with mating, building a town and attacking to other groups.

Guard system: As the name suggests, it guards a territory against enemy clans. They are supposed to keep safe zones for each clan.

Territory system: If a lineage becomes too crowded, the area around them becomes their territory. The territory is covered with walls and guards automatically. If a clan becomes too strong, they attack to other clans and if they succeed, their territory gets bigger. A bigger territory means more place to build homes and increased population.

Here you can see that there was a huge battle (many corpses on the ground), and yellow creatures won it. As you explore the map, you encounter things like this, and it feels awsome :)

Siege weapons are also act as minions, they follow you and aid you in the battle.

Catapults throw rocks, which have bonus against walls and buildings. Their attacks have a small splash radius. Balistas launch big and powerful arrows. They can hit long distance targets with accuracy. Siege towers are used when marching toward the enemy territory if it is full of archers and balistas.

This is a city which is created dynamically by a bot lineage. White boxes are meant to be walls for now... :) As you can see, they have built houses (all tents for now) and walls around.











As the settlement grows, more siege weapons and guards are assigned to guard it. Bots reproduce in the city, some of them become city guards, and some of them march to surrounding areas to create new towns. A crowded group may decide to attack an enemy city if they have enough siege weapons around.

Now that's what I call a cool game menu and theme...

Hello Riemers! Thanks for the advanced terrain tutorial! Shit just got real...

My OpenGL Adventures: 3D Particle Simulator

Canerus Physics 3D Particle Simulator with C++ & OpenGL

Hello 3D World!!

Basic planes.

Displaying inclined planes without using rotation; rather by evaluating first order equations.

Displaying basic shapes by moving on their surfaces with differential equations, instead of using a naive O(N^3) algorithm.   

Implementation of polar and spherical coordinates to be able to display spherical objects with improved performance.

The tip of this shape is supposed to be closed at infinity. With a new algorithm, we calculate new points near our viewing frustum, and display them efficiently when they enter the frustum. If we look through the end from a point, perspective gives us an almost ideal shape.

Instead of having one surface per equation, we go object oriented and start implementing object classes with attributes like color, size etc.

Objects may have more than one surface in them.

Composite objects may have more then one objects in them. Instancing becomes important.

I improved my surface traversal algorithms and adapted them to the frequently used instancing system. I made sure that nothing is calculated redundantly.

Surface based collusion detection attempts with polynomial time algorithms. I could hear the cpu crying :)

After spending some time on improving the collusion detection algorithm, I gave acceleration and velocity vectors to objects. They are now environment aware; when an object moves, it sends a notification to a controller, which transmits it to other objects near it. A collusion predictor algorithm is applied between these objects, and if they are predicted to be colliding, the main collusion detection algorithm starts checking if they collided yet.

Switching to solid objects

Improved collusion detection algorithm and bug fixes.

Implemented linear and radial vector fields and added gravitational field to objects. Acceleration calculation with centrifugal and centripetal forces. Smooth orbital movement is observed after improving the time-state algorithm.

Added mobile and colored light sources.

Detailed control menu.