11

GUI Capture and replay testing using bitmaps is mentioned here as problematic(1997): http://www.imbus.de/forschung/pie-gui-test/how-to-automate-testing-of-graphical-user-interfaces/ Model based GUI Testing (Apr 15, 2003): Uses bitmaps if no other option is available http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.167.3983&rep=rep1&type=pdf ...


5

1) us5600789, FIG. 15 (1992) Automated GUI testing in which machine 1 executes a test which monitors and controls a GUI running on machine 2. Looks pretty solid. 2) us5634002 Citations on front page look helpful. 3) US 2002/0029259 (1991) At least the key features, if not all. 4) User Interface Softbots (Sept 2003) "This thesis focuses on the concept ...


5

It really looks like the Sikuli project. Sikuli Script automates anything you see on the screen. It uses image recognition to identify and control GUI components. It is useful when there is no easy access to a GUI's internal or source code. They have two academic papers that can be useful to represent Prior Art: Sikuli: Using GUI Screenshots for ...


3

I don't agree that this is overly broad. I also don't think this is simply VNC or other remote desktop tools. Those don't do bitmap analysis/pattern recognition on a remote PC. This sounds very specific to testing, and a poor way to do it. Leading solutions in this field typically run an agent on the test target, and access the UI API directly rather than ...


2

Back in the 90's Microsoft made extensive use of a tool called MSTest (they probably still use it). among other things, it captured screen images of the program under test and analyzed them to identify controls and fields. It could make decisions based on the presence or absence of graphical values. It also simulated mouse and keyboard interactions in ...


2

First post! Found this place from /. From as early as 2007: http://www.macrogoblin.com/Featured_Bots/Bot_LotRO_Fighting_Healer_Master.aspx Note that for comparison's sake, the Tank can be considered the First Computer and the Healer can be considered Second Computer. Healer's Healthbar Pixel #1: When health is low it will trigger the LIGHT HEAL routine. ...


2

The closest implementation to something like this I can think of is something like AutoIT - http://www.autoitscript.com/site/autoit/ As far as I know, it doesn't use bitmaps. Using bitmaps for something like this would be stupid, and as far as I know, nobody has done the 'screen scrape and comparison' for general control. The closest thing I can think of ...


2

There is an industry leading test automation product that has been doing this for at least 10 years. http://www.testplant.com/eggplant/testing-tools/


1

This is a particular way of accomplishing the remote monitoring of the contents of a GUI interface involving the remote system analyzing the bit map image of the system being monitored. I think other systems get a much higher level representation of the screen and don't need to do pattern matching to know what is going on at the other end.


Only top voted, non community-wiki answers of a minimum length are eligible