SMART

From BLandWiki
Jump to: navigation, search

Written by Benjamin Land (myself) in Java and C++, SMART allows any pluggable program to run RuneScape in a sandbox so you can use your computer and automate RS at the same time. At first, the name was an acronym for "Scar Minimizing Autoing Resource Thing," but in light of current events, and the open source release, SMART now stands for "SMART Minimizing Autoing Resource Thing." But some still prefer "SMART Makes Autoing Really Tedious," depending on your outlook. SMART's source code is released under the GNU GPL v3 and is maintained by myself and whoever I give access in a GitHub hosted repository at https://github.com/BenLand100/SMART (you can report bugs there as well), releases/changelog is avaliable at the SRL Forums http://villavu.com/forum/ and the usage tracker for SMART is at http://www.benland.us/smart.php

How it Works

SMART is a conglomeration of a lot of things that probably should not work, but do, and have been exploited to their fullest potential. If you want a detailed explanation, you are out of luck, otherwise, you can get the general idea here. Check out the source in the repository if you think you're good enough to figure it out. I've been slowly going through and adding documentation to the source, as well as creating bindings for other languages.

Hijacking the Image

SMART enables any program that works with RuneScape to run entirely in the background. It captures the image of the RuneScape client and sends it via a "shared memory" chunk to some remote controlling process. This code is given below. Now that SMART can load plugins, the plugins can take responsibility for returning color data across the shared memory.

   private Thread createBlitThread() {
       try {
           canvas.setBackground(new Color(0xFE, 0xFE, 0xFE));
           final Graphics canvasGraphics = canvas.getCanvasGraphics();
           Main.debug("Replacing Canvas Drawing Surface");
           Field rasterField = BufferedImage.class.getDeclaredField("raster");
           rasterField.setAccessible(true);
           WritableRaster bufferRaster = (WritableRaster) rasterField.get(buffer);
           final int[] bufferData = ((DataBufferInt) bufferRaster.getDataBuffer()).getData();
           final BufferedImage debug = new BufferedImage(width, height, BufferedImage.TYPE_INT_RGB);
           WritableRaster debugRaster = (WritableRaster) rasterField.get(debug);
           final Graphics debugGraphics = debug.getGraphics();
           final int[] debugData = ((DataBufferInt) debugRaster.getDataBuffer()).getData();
           debugGraphics.setColor(Color.RED);
           return new Thread("Smart_Image_Transfer") {
               @Override
               public void run() {
                   int len = width*height;
                   int[] temp = new int[len];
                   try {
                       Main.debug("Transfer Thread Entered");
                       while (active) {
                           if (capture && blocking) {
                               Main.debug("Internal color data capture enabled.");
                               canvas.setBuffer(buffer);
                               sleep(refresh);
                               while (capture && blocking) {
                                   sleep(refresh);
                                   nativeBuff.rewind();
                                   nativeBuff.put(bufferData,0,len); //Oh yea, major preformance boost
                                   if (renderWhileBlocking && !minimized) {
                                       final Point p = getMousePos();
                                       if (debuggfx) {
                                           nativeDebug.rewind();
                                           for (int i = 0; i < len; ++i) {
                                               int color = nativeDebug.get();
                                               if (color != transColor) {
                                                   debugData[i] = color;
                                               } else {
                                                   debugData[i] = bufferData[i];
                                               }
                                           }
                                       } else {
                                           debugGraphics.drawImage(buffer, 0, 0, null);
                                       }
                                       debugGraphics.fillOval(p.x - 2, p.y - 2, 4, 4);
                                       canvasGraphics.drawImage(debug,0,0,null);
                                   }
                               }
                               canvas.setBuffer(null);
                           } else {
                               Main.debug("Internal color data capture disabled.");
                               canvas.setBuffer(null);
                               while (!capture || !blocking) {
                                   sleep(refresh);
                               }
                           }
                       }
                       Main.debug("Transfer Thread Exited");
                   } catch (IllegalThreadStateException e) {
                       Main.debug("Transfer Thread Died");
                   } catch (Exception e) {
                       Main.debug("Transfer Thread Died");
                       e.printStackTrace();
                       blitThread.start();
                   }
               }
           };
       } catch (Exception e) {
           throw new RuntimeException("Could not replace drawing surface");
       }
   }

Controlling User Input

SMART has an entire class devoted to keyboard and mouse control. The EventNazi class controls all input, and blocking other input is done via overriding the default event queue. SMART will accept key events while events are blocked, but this is prone to malfunction and hold down keys, so it is not recommended for normal use.

Working with Simba/SCAR

SMART provides many low level methods in order to seamlessly integrate with any script without having to modify that script. SMART also implements the Extensible Input Output System (EIOS) which abstracts all HID interaction with a window, i.e. keyboard and mouse control. Programs like Simba can use this interface to control the input devices SMART provides.

Reflection

In version 2, SMART included a low level reflection system to allow access to parts of the client that are usually only accessible to Java based bots. These methods allow static and object reflection, giving many, if not all, of the capabilities of the most effective bots, while keeping all client dependent code separate from the program itself and in an easily modifible script. All that is required is string "paths" that devine the location of valus in the client, and proper use of SMART's reflection API. The generic method that is the base of the reflection system follows, it is in the Client class. public Object getFieldObject(Object o, String path) {

   try {
       String[] parts = path.split("\\.");
       Stack<String> stack = new Stack<String>();
       Field field;
       if (o == null) {
           Class c = gameLoader.loadClass(parts[0]);
           for (int i = parts.length - 1; i > 0; i--)
               stack.push(parts[i]);
           field = c.getDeclaredField(stack.pop());
           field.setAccessible(true);
           o = field.get(null);
       } else {
           for (int i = parts.length - 1; i >= 0; i--)
               stack.push(parts[i]);
       }
       if (!stack.empty()) {
           while (!stack.empty()) {
               String theField = stack.pop();
               Class theClass = o.getClass();
               field = null;
               while (field == null && theClass != Object.class) {
                   try {
                       field = theClass.getDeclaredField(theField);
                   } catch (Exception e) {
                       try {
                           theClass = theClass.getSuperclass();
                       } catch (Exception x) { break; }
                   }
               }
               field.setAccessible(true);
               o = field.get(o);
           }
       }
       return o;
   } catch (Exception e) { System.out.println("Field not found: " + path); }
   return null;

}

Distribution

SMART is now developed in a GIT repository mirrored at https://github.com/BenLand100/SMART and is released open source under the GNU GPL v3.

Keep watch at the SMART section at SRL forums http://villavu.com/forum/forumdisplay.php?f=179 for official binary releases.

SMART is designed to be compiled under a Linux operating system with the GNU C++ compiler, Sun's Java v7 compiler, and/or ports for cross compiling to windows. The included makefile is capable of building a Linux and a Windows distribution of SMART provided the required dependencies are met. You may have to change the compiler binary names for particular targets

The GUI

Old GUI

The Smart v2 GUI

The simple GUI was designed to be as compact as possible but still effective. That being said, SMART has five GUI elements

  1. RuneScape applet
  2. Refresh Rate slider
  3. Disable Smart button
  4. Disable Graphics button
  5. Enable Debug button

The Refresh Rate slider controls both the rate the game draws at, and how fast the image is copied to SCAR. The Smart button controls whether SMART is ready to accept commands or send the image (Enabled) or if it is being used as a RuneScape client only (Disabled). The Graphics button can enable or disable the client drawing to the screen, having this disabled may provide a significant performance gain, or it may not. The Debug button will significantly slow the client, but will allow a SCAR script to draw on top of the RuneScape applet without getting erased.

Updates

Besides renaming some elements, the GUI has remained mostly unchanged. But with the addition of native plugins to SMART, so came the ability for plugins to deploy their own UI buttons.

Using SMART

SMART for the average user is very simple to use, and is well documented at the SMART sections at SRL. This will be a guide, that was never properly published, on how the scripter should use SMART and what the available methods are. Given below are the exported methods as provided through the Simba plugin interface. All methods are also exported with the stdcall calling convention with the prefix exp_ and a similar name if loading by some native interface is required. BY FAR the best course of action for learning to use SMART is to check out the source and see how it all works.

Basics

The libsmartremote*.* are Simba style plugins and also generic native plugins for launching remote SMART clients (smart.jar) which require the native libsmartjni*.* libraries. Some process will spawn a smart client, and pair to it, in order to use/control it. SMART provides an image and debug image pointer, along with IO and other functions, complete with an EIOS interface for controlling multiple clients.

Core Methods

Following is a list of methods for general SMART configuration and use. function SmartGetClients(only_unpaired: boolean): integer; function SmartClientID(idx: integer): integer; function SmartSpawnClient(remote_path, root, params: string; width, height: integer; initseq, useragent, javaargs, Plugins: string): integer; function SmartPairClient(pid: integer): boolean; function SmartKillClient(pid: integer): boolean; function SmartCurrentClient(): integer;

function SmartImageArray(): integer; function SmartDebugArray(): integer;

function SmartGetRefresh: integer; procedure SmartSetRefresh(x: integer); procedure SmartSetTransparentColor(color: integer); procedure SmartSetDebug(enabled: boolean); procedure SmartSetGraphics(enabled: boolean); procedure SmartSetEnabled(enabled: boolean); procedure SmartSetCapture(enabled: boolean); procedure SmartSetNativeButton(plugid,btnid: integer; state: boolean); function SmartActive: boolean; function SmartEnabled: boolean;

IO Methods

Following is a list of the non-EIOS IO methods. procedure SmartGetMousePos(var x, y: integer); procedure SmartHoldMouse(x, y: integer; left: boolean); procedure SmartReleaseMouse(x, y: integer; left: boolean); procedure SmartHoldMousePlus(x, y, button: integer); procedure SmartReleaseMousePlus(x, y, button: integer); procedure SmartMoveMouse(x, y: integer); procedure SmartWindMouse(x, y: integer); procedure SmartClickMouse(x, y: integer; left: boolean); procedure SmartClickMousePlus(x, y, button: integer); function SmartIsMouseButtonHeld(button: integer): boolean; procedure SmartScrollMouse(x, y, lines: integer); procedure SmartSendKeys(Text: String; keywait, keymodwait: integer); procedure SmartHoldKey(Code: Integer); procedure SmartReleaseKey(Code: Integer); function SmartIsKeyDown(Code: Integer): Boolean;

Reflection Methods

Following is a list of reflection API methods. function SmartGetFieldObject(objref: integer; path: string): integer; function SmartGetFieldBoolean(objref: integer; path: string): boolean; function SmartGetFieldLongH(objref: integer; path: string): integer; function SmartGetFieldLongL(objref: integer; path: string): integer; function SmartGetFieldInt(objref: integer; path: string): integer; function SmartGetFieldShort(objref: integer; path: string): integer; function SmartGetFieldFloat(objref: integer; path: string): extended; function SmartGetFieldDouble(objref: integer; path: string): extended; function SmartGetFieldByte(objref: integer; path: string): integer;

function SmartGetFieldArrayObject(objref: integer; path: string; index: integer): integer; function SmartGetFieldArrayInt(objref: integer; path: string; index: integer): integer; function SmartGetFieldArrayFloat(objref: integer; path: string; index: integer): extended; function SmartGetFieldArrayDouble(objref: integer; path: string; index: integer): extended; function SmartGetFieldArrayBool(objref: integer; path: string; index: integer): boolean; function SmartGetFieldArrayLongH(objref: integer; path: string; index: integer): integer; function SmartGetFieldArrayLongL(objref: integer; path: string; index: integer): integer; function SmartGetFieldArrayByte(objref: integer; path: string; index: integer): integer; function SmartGetFieldArrayShort(objref: integer; path: string; index: integer): integer; function SmartGetFieldArrayChar(objref: integer; path: string; index: integer): integer;

function SmartGetFieldArray2DObject(objref: integer; path: string; x, y: integer): integer; function SmartGetFieldArray2DInt(objref: integer; path: string; x, y: integer): integer; function SmartGetFieldArray2DDouble(objref: integer; path: string; x, y: integer): extended; function SmartGetFieldArray2DFloat(objref: integer; path: string; x, y: integer): extended; function SmartGetFieldArray2DBoolean(objref: integer; path: string; x, y: integer): boolean; function SmartGetFieldArray2DLongH(objref: integer; path: string; x, y: integer): integer; function SmartGetFieldArray2DLongL(objref: integer; path: string; x, y: integer): integer; function SmartGetFieldArray2DByte(objref: integer; path: string; x, y: integer): integer; function SmartGetFieldArray2DChar(objref: integer; path: string; x, y: integer): integer; function SmartGetFieldArray2DShort(objref: integer; path: string; x, y: integer): integer;

function SmartGetFieldArray3DObject(objref: integer; path: string; x, y, z: integer): integer; function SmartGetFieldArray3DByte(objref: integer; path: string; x, y, z: integer): integer; function SmartGetFieldArray3DChar(objref: integer; path: string; x, y, z: integer): integer; function SmartGetFieldArray3DShort(objref: integer; path: string; x, y, z: integer): integer; function SmartGetFieldArray3DInt(objref: integer; path: string; x, y, z: integer): integer; function SmartGetFieldArray3DFloat(objref: integer; path: string; x, y, z: integer): extended; function SmartGetFieldArray3DDouble(objref: integer; path: string; x, y, z: integer): extended; function SmartGetFieldArray3DBoolean(objref: integer; path: string; x, y, z: integer): boolean; function SmartGetFieldArray3DLongH(objref: integer; path: string; x, y, z: integer): integer; function SmartGetFieldArray3DLongL(objref: integer; path: string; x, y, z: integer): integer;

function SmartStringFromString(jstr: integer; str: String): integer; function SmartStringFromBytes(bytes: integer; str: String): integer; function SmartStringFromChars(chars: integer; str: String): integer;

function SmartGetFieldArraySize(objref: integer; path: string; dim: integer): integer; function SmartIsPathValid(objref: integer; path: string): boolean; function SmartIsEqual(obja, objb: integer): boolean; function SmartIsNull(obj: integer): boolean; procedure SmartFreeObject(obj: integer);

Fast(er) Color Methods

These were almost never used, although they were faster. Please see Color.cpp in the repository for the source code which remains for nostalgia's sake. This no longer exists in the HEAD of the GIT repository, and none are exported though the Simba plugin interface.

History

SMART's history is larger than the average user might expect, I believe the first release of SMART was during the Summer of 2007. In fact, I began writing it as a summer project to keep myself occupied. It took almost half a year to get SMART released to the public, quite interesting if you ask me.

Pre-Release Dev Testing

SMART began as a stand alone application that communicated with SCAR via a plugin which accessed various chunks of shared memory used as a two way communication. This was released full open source, because I really didn't expect it to expand very far, being a non-user-friendly program. The Java end used a JNI dll to create the shared memory and read commands from scar as a text stored in the memory segment, very similar to the KYAB communication I wrote using sockets, but without the sockets. Eventually I began to realize that SMART could not stand alone like that, it was too hard to use and setup, so I began to work on a totally embedded version.

Embedded SMART was first released as a proof of concept more than anything else. It was all packaged in a VC++ 6.0 dll more or less in the same form it is in now. It was one of the only C++ plugins ever used with SCAR and provided the most effective (fastest) "Silent Mouse" function ever created in scar, as well as the ability to override all of scar's color and mouse methods with only one included file. To set this version of SMART up, all you really had to do was add three lines, an include, a setup procedure, and a SCAR image redirect. Much simpler than the last version were you had to open the program, name the SMART screen, add that name into SCAR, setup the shared memory stuff, then do the other setup things.

SSRL Release and Testing

Embedded SMART first moved beyond the Dev section to the SSRL section, even though no one really knows if it even exists. It was met with enthusiasm, much testing, and many suggestions, many of which I incorporated. This was when the frame rate slider was added, and when typing while blocked first appeared. I improved some of the efficiency, and made the loading of the JVM more foolproof and computer independent as more and more people began to use it. Slowly, the SMART released today took shape, and by the end of this stage was finished. In fact, the public release is the same file used at the end of this stage, not even recompiled because I didn't save old version's source code at this point.

SRL Members Release

Once SMART was through with SSRL testing, it was released to SRL Members, and ONLY to them. This was followed with much leakage and excessive whining on the parts of many people. The best quality of SMART shown though, however, in that almost any user, no matter how ignorant, seemed to be able to get it up and running with some help from the "official" tutorial (not written by me) and other members. During this time, Jagex went on their crusade against so called "real world trading" in which they removed "unbalanced trading" from RuneScape removing all chance of serious profit (especially monetary) from the game. This, once again, lowered cheating back to its purest form, cheating for skill and pride, and gave me a chance to incorporate some of my best ideas into one of the most effective bots ever.

Creation of Version 2

While SMART gained popularity among SCAR and SRL users, I began to work towards my ultimate goal, to create a bot with the color powers of SCAR and the unsurpassed capabilities of a Java bot, without the need for client edits, or a redistribution of the program every time RuneScape updates. My first attempt of this, with Solarwind, was KYAB which more or less died because it was not well accepted and people did not trust it, plus being cumbersome to use. The most important thing I was considering was how to package the Reflection, my preferred method of "client hacking", capabilities of the program. I finally decided on writing a reflection api of sorts for SCAR, allowing all of the reflection, including manipulating paths and objects, to be done within the SCAR code. This has proven, at least in the SSRL section, to be well accepted and highly efficient with well established walking and rudimentary static object clicking already taking hold. Most importantly, I have decided to stay out of the development of the reflection system within SCAR in hopes that those who write it will learn to use it themselves, something never before done in a java bot.

Along with the addition of Reflection into SMART, I modified the GUI some to include different, more precise, names. Another highly demanded item was a type of debug display that allowed SCAR to draw on top of the RS client. This was added, however I am not really happy with it because it is so slow and kind of cumbersome to use. That being said, I do not plan on improving it unless I see more demand for it in use, not just as a "this will be cool" request.

Remaining Versions

These were no longer what I would call heavy-dev versions and mostly were bug fixes, extensions, and feature requests which are outlined in the following version history.

Versions

This information was sort of written after the fact, but it is fairly accurate. As of right now, SMART v8.2 is current.

Version 1 - First Public Release

The public release of SMART brought many loyal people to SRL and jump started the SCAR user base after taking a major hit from Jagex's assault on cheating. As far as I can tell, many people use SMART exclusively for their RuneScape cheating since it allows them to use their computer. I believe this to be one of the biggest advances that was desperately needed at this time, since now many people were forced to auto on their mains, and did not want to leave them totally unattended, at least at first. They also, however, did not want to stare at a program running their character either, since in that case they might as well do it themselves. With SMART, they could listen to the program work, and do other things at the same time, or run multiple accounts if so desired.

Version 2 - SSRL Release

After much deliberation, I released a beta copy of SMARTv2 to SSRL for testing. I expanded the reflection api hopefully as far as will ever be needed, and provided a simple reflection include for the most basic Java based function, namely walking and animation. On its first release SMART surpassed what KYAB had done publicly, and was accepted by many more people in light of Jagex's plot to ruin cheating. One of the first major scripts was a Nature Rune Crafter by Wizzup and WT-Fakawi, which I personally used to craft 500+ natures. It demonstrated how SMART could take the mundane tasks out of scripting and let the scripter focus on automating larger, more elaborate projects.

Version 3 - Rewrite

Totally redone in GNU C++ with all of it written in the Netbeans IDE. Somewhere in this version SMART was ported at the source level to Linux. This version saw many tweaks to the Java end and only a few bug fixes to the C++ end. This was the version released to the public with reflection, and many remember my numerous fast bug fixes that resulted in SMART v3 and 1/3...

Version 4 - Recovery

Jagex changed the layout of their website which required a significant overhaul of the applet loading portions of SMART. In this version I added significantly stronger encoding for the Java classes to prevent people from decompiling the still closed source SMART. This version also saw some major changes in the initial setup command taking all world management out of smart and putting it in the scripter's hands. Now only a URL to the applet page and a size was required to load SMART.

Version 5 - Open Source

SMART's source was released under the GNU GPL v3 with very minor modifications to the source itself. The largest change was that the old repository was abandoned, and Netbeans was taken out of the picture in preference of a more universal makefile. The most significant change at this point was in the name, where "SCAR" was replaced with "SMART" making it a recursive acronym and more receptive to the promising Simba project that is destined to replace SCAR.

Version 6 - Bug Fixes, etc.

This version mostly includes bug fixes, internal restructures, slight modifications and additions, and the like. There was no major changes to the UI or the user experience, but quite a bit of external work done to allow other languages and programs to use SMART's interface. This was directed mostly at python, which required cdecl exports, and there are now functioning bindings for python available in the repos. A lot of work has been directed on key-typing with the hold/release methods, and for typing through the event block which is now accomplished by clicking on the screen and typing as usual, there is some lag though.

Version 7 - Updates/OOP

Along with bug fixes and various improvements, SMART was split into two processes: Client and Controller. The Client process runs the applet while the Controller can pair with clients dynamically. This used a socket / shared memory interprocess communication (IPC) protocol. Ultimately, this change was desirable to use multiple SMART instances from one process. Reflection was removed in this version due to disuse and complications of IPC.

Version 8 - Redesign/Features

The Client end of SMART was overhauled to be a completely independent Java aplication with minimal JNI requirements. This major revision saw better stability of the pairing mechanisms, revival of reflection, and the introduction of SMART plugins which can modify SMART's appearence and also provide color information from native rendering methods like OpenGL or DirectX.