Automating OS with AutoIT and AppleScript

2022-04-26

OS Automation with AppleScript and AutoIT

TestingBot has recently added the capability to run OS automation scripts during Selenium WebDriver automation. With a grid consisting of over 3000 browser and OS combinations, TestingBot now allows QA, testers and developers to automate native OS components.

What is AppleScript?

AppleScript is a built-in scripting language, available on all macOS installations, allowing you to automate parts of the macOS operating system with scripts. Another synonym for AppleScript is osascript (Open Scripting Architecture Scripting Language), this is the name of the binary responsible of running the AppleScript files.

These scripts can be recorded with the built-in Script Editor on macOS, or coded by hand.

Its syntax is very readable, and allows for using variables, keywords and script objects.

With AppleScript automation, you can create complex workflows, automate tasks or perform basic UI automation on native components such as input fields, buttons, text boxes and other native UI components (Cocoa, Swift, and others).
You can click buttons, choose from dropdown lists, choose files from your disk and more.

AppleScript example

A simple example of an AppleScript automation would be the script below, where we open Safari and enable the developer feature.

Running this script with osascript will automatically open Safari, enable the developer feature and close Safari, without any interaction required from the user.

tell application "Safari"
launch
activate
tell application "System Events"
  set v to (value of attribute "AXMenuItemMarkChar" of
  menu item "Allow Remote Automation" of menu "Develop" of
  menu bar item "Develop" of menu bar 1 of application process "Safari")
  if v is missing value then
        click menu item "Allow Remote Automation" of
        menu "Develop" of menu bar item "Develop" of menu bar 1 of application process "Safari"
      else
        log "OK"
  end if
end tell
quit
end tell

Of course, during browser automation with Selenium WebDriver, you might want to automate other use cases, such as selecting a file in the native file upload, triggered by the browser under test.

The example below shows you how to select a file from the native upload dialog. Usually you will not need this, as WebDriver has its own method of selecting and uploading files.

tell application "System Events"
    keystroke "G" using {command down, shift down}
    delay 1
    keystroke "/path/to/your/file"
    delay 1
    keystroke return

    delay 1
    keystroke return

    delay 1
end tell

AppleScript with WebDriver

TestingBot has added a feature which allows you to use AppleScript with WebDriver in its automation cloud.

By using the JavascriptExecutor functionality that WebDriver provides, you can pass scripts through a custom command we created, tb:osascript.

For example, please see the Java snippet below, where we will display an alert during your tests

String appleScriptContent = "display alert \"Success!\"";
Map<string object> osaMap = new HashMap<>();
osaMaposaMap.put("script", appleScriptContent);
((JavascriptExecutor) driver).executeScript("tb:osascript", osaMap);</string>

You can use this at any time during your WebDriver test. The test will block until the AppleScript execution has finished.

What is AutoIT?

AutoIt is a BASIC-like scripting language which is used to automate parts of the Windows UI. AutoIT scripts allows you to simulate mouse events, send key strokes and control UI components.

All recent versions of Windows are supported, including Windows 10 and 11. TestingBot has provisioned AutoIT v3 on all its Windows desktop machines.

When you first download AutoIT, you will find an executable called AutoIt3.exe, which is the program responsible for running your AutoIT scripts. Scripts are usually saved in au3 files, in plain text.

You can use the AutoIT editor (SciTE) and its element inspector to create and record AutoIT scripts. Simply click the native UI components that you want to manipulate, and copy the element identifier from the inspector to your AutoIT automation script.

AutoIT example

Let's start with a simple AutoIT example, which will display a dialog with a message on Windows.

MsgBox(0, "My Title", "Hello from AutoIT")

To automate a file upload window, you would require some more commands, including:

ControlFocus("title", "", controlID) which puts the focus on a specific control
ControlSetText("title", "/the/path/to/your/file", controlID) which sets the text (path) to your file for the specific control
ControlClick("title", "", controlID) performs the actual action (click)

AutoIT with WebDriver

TestingBot provides a command which allows you to use AutoIT with Selenium in its browser automation cloud.

You can use the built-in JavascriptExecutor functionality which comes with WebDriver and pass scripts through a custom command we created, tb:tb:autoit.

For example, please see the Java snippet below, where we will display an alert during your tests

String autoItContent = "MsgBox(0, \"My Title\", \"Hello from AutoIT\")";
((JavascriptExecutor) driver).executeScript("tb:autoit", autoItContent);

Conclusion

In this blog post we described what AutoIT and AppleScript is, which functionality it provides and how you can use it in combination with remote browser automation.

TestingBot allows you to use both AutoIT scripts and AppleScript in its remote browser grid, allowing you to automate the most complex UI components on Windows and macOS.