Smashtest • Generate tests fast Smashtest is an open-source tool and language for rapidly generating tests. Greatly speed up your web automated testing by writing tests in a tree-like format.



Trees represent how we think when we're testing. They allow us to list all the permutations that branch off from any given point. npm i -g smashtest Multiple browsers and devices UI and API Run tests in parallel Human-readable steps Awesome live reports Run locally or in CI/CD Screenshots Sample test Open Chrome Open Firefox Open Safari Navigate to STR('site.com') Click STR('Sign In') Type STR({username:}) into STR('username box') STR({username}) is STR('joe') STR({username}) is STR('bob') STR({username}) is STR('mary') Verify success STR({username}) is STR('baduser') Verify error represents Test Case 1 Test Case 2 Test Case 3 ----------- ----------- ----------- Open Chrome Open Firefox Open Safari Navigate to 'site.com' Navigate to 'site.com' Navigate to 'site.com' Click 'Sign In' Click 'Sign In' Click 'Sign In' Type 'joe' into 'username box' Type 'joe' into 'username box' Type 'joe' into 'username box' Verify success Verify success Verify success Test Case 4 Test Case 5 Test Case 6 ----------- ----------- ----------- Open Chrome Open Firefox Open Safari Navigate to 'site.com' Navigate to 'site.com' Navigate to 'site.com' Click 'Sign In' Click 'Sign In' Click 'Sign In' Type 'bob' into 'username box' Type 'bob' into 'username box' Type 'bob' into 'username box' Verify success Verify success Verify success Test Case 7 Test Case 8 Test Case 9 ----------- ----------- ----------- Open Chrome Open Firefox Open Safari Navigate to 'site.com' Navigate to 'site.com' Navigate to 'site.com' Click 'Sign In' Click 'Sign In' Click 'Sign In' Type 'mary' into 'username box' Type 'mary' into 'username box' Type 'mary' into 'username box' Verify success Verify success Verify success Test Case 10 Test Case 11 Test Case 12 ------------ ------------ ------------ Open Chrome Open Firefox Open Safari Navigate to 'site.com' Navigate to 'site.com' Navigate to 'site.com' Click 'Sign In' Click 'Sign In' Click 'Sign In' Type 'baduser' into 'username box' Type 'baduser' into 'username box' Type 'baduser' into 'username box' Verify error Verify error Verify error which represents Test Case 1 Test Case 2 Test Case 3 ----------- ----------- ----------- let driver = await new Builder().forBrowser('chrome').build(); let driver = await new Builder().forBrowser('firefox').build(); let driver = await new Builder().forBrowser('safari').build(); await driver.get('http://site.com'); await driver.get('http://site.com'); await driver.get('http://site.com'); let signInButton = await driver.findElement(By.id('#sign-in')); let signInButton = await driver.findElement(By.id('#sign-in')); let signInButton = await driver.findElement(By.id('#sign-in')); await signInButton.click(); await signInButton.click(); await signInButton.click(); await driver.wait(until.elementLocated(By.id('#username-box')), 10000); await driver.wait(until.elementLocated(By.id('#username-box')), 10000); await driver.wait(until.elementLocated(By.id('#username-box')), 10000); let usernameBox = await driver.findElement(By.id('#username-box')); let usernameBox = await driver.findElement(By.id('#username-box')); let usernameBox = await driver.findElement(By.id('#username-box')); await usernameBox.sendKeys('joe'); await usernameBox.sendKeys('joe'); await usernameBox.sendKeys('joe'); await driver.wait(until.elementLocated(By.id('#success-element')), 10000); await driver.wait(until.elementLocated(By.id('#success-element')), 10000); await driver.wait(until.elementLocated(By.id('#success-element')), 10000); Test Case 4 Test Case 5 Test Case 6 ----------- ----------- ----------- let driver = await new Builder().forBrowser('chrome').build(); let driver = await new Builder().forBrowser('firefox').build(); let driver = await new Builder().forBrowser('safari').build(); await driver.get('http://site.com'); await driver.get('http://site.com'); await driver.get('http://site.com'); let signInButton = await driver.findElement(By.id('#sign-in')); let signInButton = await driver.findElement(By.id('#sign-in')); let signInButton = await driver.findElement(By.id('#sign-in')); await signInButton.click(); await signInButton.click(); await signInButton.click(); await driver.wait(until.elementLocated(By.id('#username-box')), 10000); await driver.wait(until.elementLocated(By.id('#username-box')), 10000); await driver.wait(until.elementLocated(By.id('#username-box')), 10000); let usernameBox = await driver.findElement(By.id('#username-box')); let usernameBox = await driver.findElement(By.id('#username-box')); let usernameBox = await driver.findElement(By.id('#username-box')); await usernameBox.sendKeys('bob'); await usernameBox.sendKeys('bob'); await usernameBox.sendKeys('bob'); await driver.wait(until.elementLocated(By.id('#success-element')), 10000); await driver.wait(until.elementLocated(By.id('#success-element')), 10000); await driver.wait(until.elementLocated(By.id('#success-element')), 10000); Test Case 7 Test Case 8 Test Case 9 ----------- ----------- ----------- let driver = await new Builder().forBrowser('chrome').build(); let driver = await new Builder().forBrowser('firefox').build(); let driver = await new Builder().forBrowser('safari').build(); await driver.get('http://site.com'); await driver.get('http://site.com'); await driver.get('http://site.com'); let signInButton = await driver.findElement(By.id('#sign-in')); let signInButton = await driver.findElement(By.id('#sign-in')); let signInButton = await driver.findElement(By.id('#sign-in')); await signInButton.click(); await signInButton.click(); await signInButton.click(); await driver.wait(until.elementLocated(By.id('#username-box')), 10000); await driver.wait(until.elementLocated(By.id('#username-box')), 10000); await driver.wait(until.elementLocated(By.id('#username-box')), 10000); let usernameBox = await driver.findElement(By.id('#username-box')); let usernameBox = await driver.findElement(By.id('#username-box')); let usernameBox = await driver.findElement(By.id('#username-box')); await usernameBox.sendKeys('mary'); await usernameBox.sendKeys('mary'); await usernameBox.sendKeys('mary'); await driver.wait(until.elementLocated(By.id('#success-element')), 10000); await driver.wait(until.elementLocated(By.id('#success-element')), 10000); await driver.wait(until.elementLocated(By.id('#success-element')), 10000); Test Case 10 Test Case 11 Test Case 12 ------------ ------------ ------------ let driver = await new Builder().forBrowser('chrome').build(); let driver = await new Builder().forBrowser('firefox').build(); let driver = await new Builder().forBrowser('safari').build(); await driver.get('http://site.com'); await driver.get('http://site.com'); await driver.get('http://site.com'); let signInButton = await driver.findElement(By.id('#sign-in')); let signInButton = await driver.findElement(By.id('#sign-in')); let signInButton = await driver.findElement(By.id('#sign-in')); await signInButton.click(); await signInButton.click(); await signInButton.click(); await driver.wait(until.elementLocated(By.id('#username-box')), 10000); await driver.wait(until.elementLocated(By.id('#username-box')), 10000); await driver.wait(until.elementLocated(By.id('#username-box')), 10000); let usernameBox = await driver.findElement(By.id('#username-box')); let usernameBox = await driver.findElement(By.id('#username-box')); let usernameBox = await driver.findElement(By.id('#username-box')); await usernameBox.sendKeys('baduser'); await usernameBox.sendKeys('baduser'); await usernameBox.sendKeys('baduser'); await driver.wait(until.elementLocated(By.id('#error-element')), 10000); await driver.wait(until.elementLocated(By.id('#error-element')), 10000); await driver.wait(until.elementLocated(By.id('#error-element')), 10000);

Setup 1. Install NodeJS Make sure you have NodeJS installed. We recommend the LTS version. Use node -v to check. 2. Install Selenium Webdriver (if you're doing web UI testing) Make sure all of the browsers you want to automate are installed Make sure you have Java installed. Use java -showversion to check. Choose one of the following options: Option 1: Webdriver manager Webdriver managers take care of the installation process for you, but usually require a second console to be open during test runs, require an additional command-line flag to be passed in, and might not have support for less-popular browsers, such as Safari. You have to run a single update command when a browser has a major release. Mac users, consider using webdriver-manager, which supports Chrome and Firefox: In a new console, run npm install -g webdriver-manager (if that gives you permissions errors, put sudo before npm). Run webdriver-manager update to download the latest versions of everything. Run webdriver-manager start . This must always be running when executing tests.

Windows users, consider using selenium-standalone, which supports Chrome, Firefox, IE, and Edge: In a new console, run npm install -g selenium-standalone (if that gives you permissions errors, put sudo before npm). Run selenium-standalone install to download the latest versions of everything. Run selenium-standalone start . This must always be running when executing tests.

Whenever running Smashtest, always run it as smashtest --test-server=http://localhost:4444/wd/hub (or whatever the port is), or include that flag in the config file. If you upgrade your browser, especially Chrome, make sure you run the update/install command (step 2) to make sure your drivers are in sync with the browser versions you have installed. Option 2: Manual install This option allows you to run everything from just one console, handles any browser, and doesn't require you to pass in additional command-line flags, but the install takes longer, and you have to do a manual update when a browser has a major release. You'll need to download individual executables (called "drivers") for each browser you want to automate: Chrome • Firefox • Edge • IE Note: Safari 10+ on MacOS comes pre-installed with SafariDriver Each driver executable should be placed in your system's PATH On MacOS that's usually /usr/local/bin (run echo $PATH to find out)

(run to find out) On Windows that's usually C:\Program Files (run echo %PATH% to find out) Download the latest version of selenium standalone. For example, click the 3.9 folder, then download selenium-server-standalone-3.9.1.jar. Set the SELENIUM_SERVER_JAR environment variable to the absolute path of the jar file you just downloaded. On MacOS add the line export SELENIUM_SERVER_JAR=[path here] to the file ~/.bash_profile , then restart your console

to the file , then restart your console On Windows run setx -m SELENIUM_SERVER_JAR "[path here]" (you may have to start a second command prompt as administrator), then restart your original command prompt If you upgrade your browser, especially Chrome, make sure you re-download and install the correct driver versions. Option 3: Cloud service or Grid Alternatively, you can run Smashtest with a Selenium Grid. This is an advanced, distributed configuration. To use a grid from a cloud service (Sauce Labs, BrowserStack, etc.): Set up the capabilities in your test. See your cloud service documentation for details. Set --test-server to your cloud service's url, or include that flag in the config file.

To install a grid locally: Follow steps 1-4 under Option 2 (but do not set the SELENIUM_SERVER_JAR variable) In a console, run java -jar selenium-server-standalone-[version].jar -role hub and keep it running In another console, run java -jar selenium-server-standalone-[version].jar -role node -hub http://localhost:4444/grid/register and keep it running Whenever running Smashtest, always run it as smashtest --test-server=http://localhost:4444/wd/hub (or whatever the port is), or include that flag in the config file See the grid page for more information.

3. Install Smashtest From a console, run npm install -g smashtest (if that gives you permissions errors, put sudo before npm) 4. Install a grammar (highly recommended) A grammar will highlight your code, making your .smash files pretty and much easier to read. They are currently available on the Atom and VSCode editors. Atom Download Atom Install the Smashtest package Set configuration for smash files Ctrl/Cmd + Shift + P Type "open config" and hit enter (this will open your config.cson file) Add the following to the bottom of the file: ".smash.source": editor: autoIndentOnPaste: false commentStart: "//" tabLength: 4 VSCode Download VSCode Install the Smashtest extension

Writing your first test Here's what to do Create a new text file called "helloworld.smash" Put the following into the file: Open Chrome Navigate to STR('google.com') Type STR('hello world[enter]') into STR('textbox') Open a console and cd to that file's directory Run smashtest Watch test run Open live report while test is running (see console output for path) So what did I just do? Smashtest comes with a lot of built-in steps, a few of which you just used. Think of steps as being arranged like comments on a website. Just like comments are indented to the comment they're replying to, steps are indented to the step they come after. In this example, there's only one line of steps, hence there's only one test. We call tests branches. So let's add a branch! Open Chrome Navigate to STR('google.com') Type STR('hello world[enter]') into STR('textbox') Type STR('hello universe[enter]') into STR('textbox') Notice line 7. It's at the same indent level as line 5. That means both lines will follow line 3. Therefore, this file will have two branches: Open Chrome, Navigate to 'google.com', Type 'hello world[enter]' into 'textbox' Open Chrome, Navigate to 'google.com', Type 'hello universe[enter]' into 'textbox' Try running it yourself. Then, give yourself a pat on the back!

Debug and learn Try this debugging technique... Open Chrome Navigate to STR('google.com') MOD(~ Type 'hello world[enter]' into 'textbox') Type STR('hello universe[enter]') into STR('textbox') Put a MOD(~) in front of line 5. That single branch will be isolated, the browser will run non-headless (you can see it), and the test will pause before that line. You can now use the console to try out steps, move to the next step, repeat a previous step, etc. This mode is called the REPL. It's great for learning and debugging. Development technique Using MOD(~) is actually a recommended test development technique: Write a step Put a MOD(~) at the end of it Run smashtest, which runs the browser to that line Come up with all the permutations that can branch off from that point (using the browser as a guide) List them as steps, indented to the step with the MOD(~) Repeat Examples Feel free to play with this UI test example and this API test example.

Basic language syntax Branches Open Chrome COM(// executed in both branches) Navigate to STR('site.com') COM(// executed in both branches) Click STR('one') COM(// branch 1 ends here) Click STR('two') COM(// branch 2 ends here) COM(// produces branches:) COM(// 1CLOSEP open, nav, click 'one') COM(// 2CLOSEP open, nav, click 'two') Open Chrome Navigate to STR('google.com') Do a search COM(// this step ends branch 1) Navigate to STR('pets.com') Buy cat food COM(// this step ends branch 2) Buy dog food COM(// this step ends branch 3) Step blocks Open Chrome COM(// this group of 5 steps is known as a "step block" (same indent, no blank lines in betweenCLOSEP) Open Firefox Open Safari Open Edge Open IE COM(// one empty line under a step block is mandatory) Navigate to STR('site.com') COM(// 5 separate branches end in this step) Sequential MOD(..) Open Chrome Nav to STR('site.com') Click STR('button') COM(// is the same as) Open Chrome Nav to STR('site.com') Click STR('button') Textual steps This step is a function call COM(// it executes an action) MOD(-) This step is a textual step COM(// it's just a piece of text to organize your tests) Look, I can put the "-" modifier at the end too! MOD(-) Navigate to STR('site.com') MOD(-) Logged-in tests COM(// etc.) MOD(-) Logged-out tests COM(// etc.) Code blocks Open Chrome Navigate to STR('site.com') Click the logo { COM(// this is a code block) COM(// you can do anything js or nodejs supports) (JSKEYWORD(await) JSFUNC($)(STR('#logo'))).JSFUNC(click)(); JSVARIABLE(browser).JSVARIABLE(driver); COM(// webdriverjs's WebDriver object ) } COM(// must end at the same indent level as the starting line) Functions Open Chrome Navigate to STR('site.com') Click STR('element') COM(// all 3 steps are function calls to built-in ("packaged"CLOSEP functions) FD(* Log In) COM(// this is a function declaration) Click STR('log in box') Type STR('username') into STR('username box') Click STR('ok') Open Chrome Navigate to STR('site.com') Log In COM(// this is a function call (it will execute the 3 login stepsCLOSEP) Log Out COM(// another function call) log out COM(// steps are case insensitive) FD(* Log Out) { COM(// this is a code block) (JSKEYWORD(await) JSFUNC($)(STR('.logout-button'))).JSFUNC(click)(); } Variables STR({username}) = STR('superman') COM(// this sets the global variable 'superman') STR({username}) is STR('superman') COM(// same as above) STR({username}) is STR("superman") COM(// same as above) STR({username}) is STR([superman]) COM(// same as above) Type STR('{username} is a handsome guy') into STR('textbox') ElementFinders Open Chrome Navigate to STR('https://www.site.com') On homepage { COM(// describe props, which are things on the page and/or the state of those things) JSFUNC(props)({ STR('message box'): STR(`.textbox, enabled`), COM(// inside the `` is an ElementFinder ,) COM(// a special syntax for finding elements) STR('login button'): STR(`#login`), STR('search results'): STR(`) STR(#list) STR(.result, 'one') STR(.result, 'two') STR(`), STR('groovy'): STR(`'contains this groovy text'`), STR('retro button'): STR(`selector '.button', groovy`) }CLOSEP; } Type STR('hello world') into STR('message box') COM(// using a prop makes it easier to read and refactor)

Examples Web UI TodoMVC tests • what's being tested REST API OnWater API tests • what's being tested How to read and run main.smash is the main entry point for tests in both examples. To run an example, clone the project, then run smashtest inside the example's directory.

Running Tests Command line Run smashtest [files] [options] [files] can be a filename or a glob (e.g., tests/*.smash)

[options] are explained here. All files are concatenated into one big piece of text, and everything at indent 0 (e.g., function declarations) is accessible to all the other files. After branches are compiled, they are all run. smashtest will run all .smash files in the current directory (but not sub-directories, unless --recursive is set). Ordering of branches Branches are run in order of frequency, high to low. Otherwise, branches are shuffled randomly. Here's why: You can stop the run at any time and get a good sampling of different functionality Diversifies the browsers and devices used at any given time, so as to maximize the capabilities of a grid (such as selenium grid) Disable with the --random=false flag. Headless For UI testing, browsers will run headless by default, except if you're debugging (in which case they will run visibly). Safari, IE, and Edge don't support headless, so they always run visibly. Override with the --headless=[true/false] flag.

Command-line options Command-line flags vs. config file Pass in options via command-line flags (e.g., smashtest test.smash --headless=false --max-parallel=7) or by setting them in a config file. The config file must be in the same directory where smashtest is called from, must be valid json, and must be called smashtest.json: { JSVARIABLE("headless"): JSCONST(false), JSVARIABLE("max-parallel"): JSCONST(7) } Command-line flags will override config file options when they conflict. List of options --debug=[hash] Only run the branch with the given hash, in debug mode. Ignore $'s, ~'s, groups, and frequency. --groups=[value] or --groups="group1,group2+group3" Only run branches that are part of one of the groups covered by the expression. Some group names are set automatically. For example, you can do --groups=chrome, --groups=firefox, --groups=safari, --groups=ie, --groups=edge to only run branches that use those browsers. + = AND, , = OR, where AND takes precendence. In other words, --groups="one,two+three,four" means run branches part of group one, or both two and three, or four. --g:[name]=[value] Sets a global variable before every branch. --headless=[true/false] Whether to run browsers as headless. Overrides default headless behavior (which is headless unless debugging). --help or -? Show a help prompt --max-parallel=[N] Do not run more than N branches simultaneously. 5 by default. If you're hitting a selenium grid (i.e., with --test-server), set this high enough so as to max out the available slots (capabilities). Tests will block on Open [browser] if a capability isn't available yet. --max-screenshots=[N] Do not take more than N screenshots. No limit by default. Screenshots taken but deleted after a branch passed do not count against N. --min-frequency=[high/med/low] Only run branches at or above this frequency. Set to "med" if omitted. --no-debug Fail if there are any $'s or ~'s. Useful to prevent debugging in CI. --output-errors=[true/false] Whether to output all errors to console. True by default. Good for when you're developing tests and expect a few to fail (goes well with -s). Also good for unit tests that run quick and you don't want to switch to the report each time. --p:[name]=[value] Sets a persistent variable. --random=[true/false] Whether to randomize the order of branches. True by default. --recursive Scan the current directory and subdirectories for .smash files (when no filenames are passed in). Without this flag, only the current directory is scanned. --repl or -r Opens the REPL (drive Smashtest from command line) --report-domain=[domain or domain:port] Domain where the report server should run. This is where the report page gets its live updates. Port indicates what port smashtest should run on. If omitted, an open port will be chosen, starting with port 9000. Domain indicates the domain of the machine you're running smashtest on. When this option is omitted, localhost is chosen by default. Choose a domain other than localhost for when you're running tests in CI and want people to hit the report externally. --report-history=[true/false] Whether to keep a history of all reports by date/time. If true, will output each report and passed-data file to smashtest/reports/[datetime]/. Otherwise, will output each report and passed-data to smashtest/, possibly overriding the ones from the previous run. --report-path="[absolute path]" Sets a custom absolute path for the report and passed-data. Normally these files are outputted to smashtest/ (from the current directory), but if this flag is set, they will be outputted to [absolute path]/smashtest/. --report-server=[true/false] Whether to run a server during run for live report updates. Default is true. --screenshots=[true/false] Whether to take screenshots before and after each step. Default is false. --skip-passed=[true/false/filename], -s, -a Doesn't run branches that passed last time. Carries them and their passed state over into the new report. Useful when you're fixing a breaking test and don't want to rerun the whole suite again, or when you only want to run new or updated tests. --skip-passed=true don't run branches that passed last time (carry them over), and look to ./smashtest/passed-data for the data



--skip-passed=false run all branches that are supposed to run



--skip-passed=[filename] don't run branches that passed last time, and use the given file as the record



-s same as --skip-passed=true



-a same as --skip-passed=false Consider copying smashtest/passed-data to a different filename from time-to-time in order to save it. Then restore it by copying that file back in. This is because every new run will overwrite whatever's in smashtest/passed-data. --step-data=[all/fail/none] Keep step data for all steps, only failed steps, or no steps. Set to 'all' by default. Includes screenshots. Helps reduce size of reports. --test-server=[url] Location of the test server (e.g., http://localhost:4444/wd/hub for selenium server). --version or -v Outputs the version of Smashtest. More memory If you need more memory, set the NODE_OPTIONS nodejs variable. For example, in MacOS you'd enter export NODE_OPTIONS="--max_old_space_size=[max MB to use]" && smashtest [files]

Selective test running -s You can skip running branches that passed in the previous run by running smashtest -s or by using the --skip-passed=true flag. Useful when you're fixing a breaking test and don't want to rerun the whole suite again, or when you only want to run new or updated tests. Only modifier ($) Open Chrome Navigate to STR('google.com') Do a search DEBUG($ Navigate to 'pets.com') COM(// only runs branches that pass through this step) Buy cat food COM(// i.e., the branch ending here) Open Chrome DEBUG($ Open Firefox) Open Safari Desktop DEBUG($ Mobile) Navigate to STR('google.com') Do a search COM(// only runs the Firefox mobile branch that ends here) Open Chrome DEBUG(Open Firefox $) Open Safari Desktop DEBUG(Mobile $) DEBUG(Navigate to 'google.com' ~) COM(// use to help isolate a branch for a debug) Groups Open Chrome Open Firefox Navigate to STR('google.com') MOD(#google) MOD(#happy-path) Do a search Navigate to STR('pets.com') MOD(#pets) MOD(#happy-path) Buy cat food Only run Google branch: smashtest --groups=google Only run branches in happy path OR pets: smashtest --groups="happy-path,pets" Only run branches in happy path AND pets: smashtest --groups="happy-path+pets" Only run Firefox branches: smashtest --groups=firefox (built-in group) Only run branches in happy path AND pets, or firefox AND pets: smashtest --groups="happy-path+pets,firefox+pets" Frequency Open Chrome Navigate to STR('google.com') Do something you want tested very often MOD(#high) COM(// good for quick smoke tests) Do something you want usually tested MOD(#med) COM(// your normal test suite) Do something you want usually tested COM(// #med is the default freq of a branch if #high/med/low is omitted) Do something you want tested once in a while MOD(#low) COM(// good for long-running, low-risk, edge-casey stuff) This branch will be med MOD(#med) MOD(#some-group) COM(// the later step controls the branch's freq) This branch will be low Run low/med/high tests: smashtest --min-frequency=low Run med/high tests: smashtest --min-frequency=med Run high tests: smashtest --min-frequency=high Branches are also run in order of frequency, from high to low (and are shuffled within their freq).



MOD(#desktop) , MOD(#mobile) ,

MOD(#chrome) , MOD(#firefox) , MOD(#safari) , MOD(#ie) , MOD(#edge) ,

MOD(#low) , MOD(#med) , MOD(#high) These are the reserved hashtags used by smashtest

CI/CD integration Test server If you're running your tests from a test server, such as selenium grid, include smashtest --test-server=[url] with the test server's url. A selenium server, for example, runs on http://localhost:4444/wd/hub by default. If you're using a cloud service, be sure to set your capabilities in your tests as well. Report server Reports are outputted to ./smashtest/report.html from the directory smashtest is run from. Make sure users who access the report have access to the contents of the ./smashtest directory. To ensure the report can receive live updates during a run, set smashtest --report-domain=[domain:port] where the domain is of the box running smashtest, and port is the port where you want report apis to be accessible. Flakiness After running smashtest, run smashtest -s one or more times to rerun failed tests (mitigating environmental or selenium flakiness). Or, consider running smashtest -s --screenshots=true, so that failed tests always get rerun with screenshots on every step. Exit codes The smashtest process exits with exit code 1 if at least one branch failed, 0 otherwise. No debug Run smashtest --no-debug to fail run if a MOD(~), MOD(~~), MOD($), exists anywhere in your tests. This way, you're certain you're running the full suite (and nobody accidentally committed their local debug modifiers).

Reports Location The live report is outputted to smashtest/report.html from the directory where smashtest was run. If --report-history=true is set, the report will be at smashtest/reports/smashtest-[timestamp]/report.html. Failed branches Similar failed branches are grouped together. This not only shortens the report, it also helps you debug. For example, if upon expanding a group you see that the same test fails regardless of browser, you know the browser isn't the culprit. Colors In the report, steps that execute code are colored as passed, failed, running, skipped, or not run yet, while non-executable textual steps and functions are colored in plain gray. Branches themselves have similar colors. What's in a name? We believe that tests shouldn't be named. Tests have a tendency to look very similar to other tests, and it's actually quite common to have branches that only differ by one step. Names tend to just be a crude summary of one or two steps (e.g., "logged-in cart test 016"). Having to think of a name only slows you down. Smashtest identifies branches by a unique hash, and just shows a list of steps in the report, collapsed by similarity to keep things clean. If you really want to name a test, use textual steps. Performance constraints Due to performance constraints, reports are limited to 500 branches of each type (passed, failed, etc.), and currently running is limited to 20.

Language Indents Smash, the language that's run by smashtest, uses exactly 4 spaces for each indent (no tabs). Each step is indented to the step it follows (like comments on a website). Blank lines Blank lines are generally ignored. Use them for stylistic organization, or to group similar steps. The only time they matter is in a step block. A step block cannot have blank lines between members, and must have one blank line below (if it has children). You can also use blank lines to prevent a step block from forming. Log in as STR('joe') COM(// this is a simple step block) Log in as STR('bob') Log in as STR('mary') Do test stuff Scope All files passed to smashtest will be concatenated into one long piece of text at runtime. Everything at indent 0 (e.g., function declarations) will be accessible in all other files. For example: COM(// file1.smash) COM(// -----------) Open Chrome Do test stuff COM(// declared in file2.smash) COM(// file2.smash) COM(// -----------) FD(* Do test stuff) Navigate to STR('site.com') Click STR('button') Modifiers Modifiers are symbols that come before or after the step: MOD(! - +) This step is surrounded by modifiers MOD(~ $ #med -s) Each modifier must be surrounded by spaces. The only modifier which acts differently based on if it comes before vs. after the step is MOD(~)

Step blocks Simple step blocks Log in as STR('joe') COM(// this group of 3 steps is a step block) Log in as STR('bob') Log in as STR('mary') COM(// one empty line under a step block is mandatory) Do test stuff COM(// produces branches:) COM(// 1CLOSEP log in as joe, do test stuff) COM(// 2CLOSEP log in as bob, do test stuff) COM(// 3CLOSEP log in as mary, do test stuff) Vertical list of 2 or more steps at the same indent, no blank lines in between. Must end in an empty line if it has children. Multi-level step blocks Anon open chrome nav to STR('searchengine.com') [ type STR('hello world[enter]') into STR('search box') type STR('hello world') into STR('search box') click STR('search') ] verify search results COM(// called after each leaf in the bracketed branches) COM(// produces branches:) COM(// 1CLOSEP open, nav, type w/ enter, verify) COM(// 2CLOSEP open, nav, type, click, verify) Named open chrome nav to STR('searchengine.com') enter search terms [ COM(// same as the first example, but names the step block) type STR('hello world[enter]') into STR('search box') type STR('hello world') into STR('search box') click STR('search') ] verify search results Think of square brackets as a "big parenthesis" around multiple steps. A named step block will look like a function call in the report. They keep things neat. Always put modifiers before the [. With code blocks COM(// these 3 steps form a step block, even though they have code blocks) Click one { (JSKEYWORD(await) JSFUNC($)(STR('#one'))).JSFUNC(click)(); } Click two { (JSKEYWORD(await) JSFUNC($)(STR('#two'))).JSFUNC(click)(); } Click three { (JSKEYWORD(await) JSFUNC($)(STR('#three'))).JSFUNC(click)(); } Verify action was completed Sequential MOD(..) Open Chrome Nav to STR('site.com') Click STR('button') COM(// is the same as) Open Chrome Nav to STR('site.com') Click STR('button') More on the sequential modifier.

Textual steps (-) MOD(-) Tests with a logged-in user COM(// a textual step) Log in as STR('bob') Log in as STR('mary') COM(// etc.) MOD(-) Tests with invalid users COM(// a textual step) Log in as STR('') Log in as STR('baduser') COM(// etc.) Textual steps serve to mark and organize. They don't execute anything. They're gray in the report (to differentiate them from executable steps). Uses Can be a heading: grouping and describing the steps that come after and/or the whole branch Alternatively, use functions or named step blocks to group multiple steps into one named task. They will appear as collapsible sections in the report.



Can be a comment you want to appear in the report For example, to describe the state of the app or test at that moment in time (or a state that's beginning or ending) A requirement, description, id, or name Describe a manual step



Can be used to "comment out" an executable step (though it's recommended to use MOD(-s) or COM(//) in this case) Recommended test structure Textual steps are great for dividing tests based on category. In this example, the app is divided and further sub-divided into functionalities, ending in function calls that are implemented in another file. Notice how the "When" steps correspond to the category, while the "Givens" and "Thens" vary from test to test. Given I am at my note-taking app MOD(-) Notes MOD(-) Creating MOD(-) with a normal string Given no notes exist Given notes exist When a note is created Then it is properly displayed MOD(-) with a whitespace-only string When a note is created with an empty string When a note is created with whitespace only Then no note is created COM(// ...) MOD(-) Updating COM(// ...) MOD(-) Deleting COM(// ...) MOD(-) Login COM(// ...) MOD(-) Register COM(// ...) MOD(-) Search COM(// ...)

Functions (*, **) Function calls Function call here Function call with STR('strings') and STR("strings") and STR([strings]) as inputs Function call with STR({variables}) and STR({{variables}}) as inputs Function call with STR('{variables} inside strings') as inputs Function declarations Public FD(* Function declaration here) Navigate to STR('site.com') Check STR('checkbox') Function declaration here COM(// function call, runs the nav and check steps above) A function declaration is accessible to all steps under its parent (or all steps, if declared at indent 0). With inputs FD(* Function declaration that STR({{takes}}) in STR({{inputs}})) Navigate to STR('site.com/{{takes}}/{{inputs}}') Check STR('checkbox') Function declaration that STR('string input') in STR({var input}) COM(// function call) With brackets FD(* Log In) [ COM(// optional brackets) Click STR('login button') Type STR('username') into STR('username box') Click STR('sign in button') ] With code block FD(* Log In) { (JSKEYWORD(await) JSFUNC($)(STR('.login-button'))).JSFUNC(click)(); (JSKEYWORD(await) JSFUNC($)(STR('.username'))).JSFUNC(sendKeys)(STR('username')); (JSKEYWORD(await) JSFUNC($)(STR('.signin-button'))).JSFUNC(click)(); } FD(* Log In) { (JSKEYWORD(await) JSFUNC($)(STR('.login-button'))).JSFUNC(click)(); } Type STR('username') into STR('username box') Click sign-in { (JSKEYWORD(await) JSFUNC($)(STR('.signin-button'))).JSFUNC(click)(); } Multiple branches FD(* Nav to the cart page) COM(// has 2 branches, which represent 2 ways of doing this thing) Navigate to STR('/') Click STR('cart button') Navigate to STR('/cart') Nav to the cart page Click STR('checkout') COM(// produces branches:) COM(// 1CLOSEP nav to /, click cart, click checkout) COM(// 2CLOSEP nav to /cart, click checkout) FD(* Choose a browser) Open Chrome Open Firefox FD(* Choose a viewport) Desktop Mobile Choose a browser Choose a viewport Do a test COM(// produces branches:) COM(// 1CLOSEP open chrome, desktop, do a test) COM(// 2CLOSEP open firefox, desktop, do a test) COM(// 3CLOSEP open chrome, mobile, do a test) COM(// 4CLOSEP open firefox, mobile, do a test) Declarations inside declarations (calling in context) FD(* On Desktop) COM(// when this is called, all child function declarations are made accessible to future steps) FD(* On Homepage) FD(* Logout) COM(// logout actions for desktop homepage) FD(* On Cart page) FD(* Logout) COM(// logout actions for desktop cart page) FD(* On Mobile) FD(* On Homepage) FD(* Logout) COM(// logout actions for mobile homepage) FD(* On Cart page) FD(* Logout) COM(// logout actions for mobile cart page) FD(* On Desktop) Desktop COM(// built-in step for desktop viewport) FD(* On Mobile) Mobile COM(// built-in step for mobile viewport) MOD(..) Open Chrome On Desktop Navigate to STR('/cart') On Cart page Logout COM(// executes the logout for the desktop cart page) Gherkin FD(* I log in) COM(// matches all 4 function calls below) COM(// etc.) Given I log in COM(// if an exact match cannot be found, gherkin (given/when/then/andCLOSEP is stripped) When I log in Then I log in And I log in {var} = F FD(* Choose a username) STR({x}) = STR('bob') STR({x}) = STR('joe') STR({x}) = STR('mary') STR({username}) = Choose a username Type STR({username}) into STR('username box') COM(// produces branches:) COM(// 1CLOSEP type 'bob') COM(// 2CLOSEP type 'joe') COM(// 3CLOSEP type 'mary') Private FD(* On cart page) FD(** Private function) COM(// not made accessible after a call to "On cart page") COM(// etc.) Private function COM(// call is ok) Something Private function COM(// call is ok) On cart page COM(Private function) COM(// compile-time error) Hooks FD(*** Before Every Branch) { COM(// stuff to do before every branch begins) } More on hooks, and a full list of supported ones. Patterns Encapsulating and refactoring FD(* Order dinner) COM(// multiple branches for different ways of accomplishing the same thing) MOD(-) Variant 1 Add beans to meal Add rice to meal MOD(-) Variant 2 Add rice to meal Add beans to meal Organizing COM(// main-tests.smash - bird's eye view of all tests helps ensure we have full coverage) COM(// ----------------) MOD(-) Test app MOD(-) Homepage tests Display homepage test MOD(-) Cart tests Empty cart test Full cart test MOD(-) Search tests Empty search test Base case search test COM(// cart-tests.smash) COM(// ----------------) FD(* Empty cart test) COM(// etc.) FD(* Full cart test) COM(// etc.) Dividing a single declaration into multiple files COM(// logout.smash) COM(// ------------) FD(* On Desktop) COM(// this func declaration split into multiple files (to keep logout togetherCLOSEP) FD(* On Homepage) FD(* Logout) COM(// etc.) FD(* On Mobile) FD(* On Homepage) FD(* Logout) COM(// etc.) COM(// search.smash) COM(// ------------) FD(* On Desktop) COM(// this func declaration split into multiple files (to keep search togetherCLOSEP) FD(* On Homepage) FD(* Do a search) COM(// etc.) FD(* On Mobile) FD(* On Homepage) FD(* Do a search) COM(// etc.) "On" pattern COM(// Call a function starting with "On" to indicate that this is the current state) COM(// (and not an action to be performed, such as navigating thereCLOSEP) COM(// e.g., "On [page]" can do verifications and set up props/functions for that page) FD(* On cart page) { COM(// call when on the cart page) COM(// set up ElementFinder props for everything on this page) JSFUNC(props)({ STR('list of items'): STR(`) STR(#list) STR(.item) STR(.item) STR(.item) STR(`), STR('checkout button'): STR(`#checkout`) }); } COM(// do some initial page verifications) Verify at page STR('/cart') Verify STR('list of items') is visible COM(// expose cart-related functions) FD(* Add item to cart) COM(// etc.) Open Chrome Nav to STR('/cart') On cart page Add item to cart Enforce permutations COM(// this pattern ensures that all permutations of these function calls are implemented below) On Desktop On Mobile Logged In Logged Out Verify something COM(// compile-time error if we're missing a permutation here:) FD(* On Desktop) FD(* Logged In) FD(* Verify something) COM(// etc.) FD(* Logged Out) FD(* Verify something) COM(// etc.) FD(* On Mobile) FD(* Logged In) FD(* Verify something) COM(// etc.) FD(* Logged Out) FD(* Verify something) COM(// etc.) FD(* Try every string permutation) COM(// calling this function ensures that all 3 permutations) String is empty COM(// are implemented directly below, so you don't forget) String is whitespace String is normal COM(// in another file...) Type STR({string)LC(:)STR(}) into STR('textbox') Try every string permutation COM(// error if any of the 3 are missing) FD(* String is empty) STR({string}) = STR('') COM(// ...) FD(* String is whitespace) STR({string}) = STR(' ') COM(// ...) FD(* String is normal) STR({string}) = STR('normal') COM(// ...) Rules for matching calls to declarations Simple case Open Chrome COM(// 4CLOSEP looks among children of this step, finds line 14) Navigate to STR('/page1') COM(// 3CLOSEP looks among children of this step, finds nothing) Login COM(// 2CLOSEP looks among children of this step, finds nothing) My function COM(// 1CLOSEP function call *** START HERE ***) Click STR('something') Click STR('something else') Click STR('something else') Navigate to STR('/page2') Login FD(* My function) COM(// the one that's matched ("overrides" line 17CLOSEP) COM(// etc.) FD(* My function) COM(// etc.) For every function call, searches for a matching function declaration among the children of that call's parent. If nothing is found, searches among the grandparent's children, then the great-grandparent's children, etc. until it searches among all declarations at indent 0 before erroring out. Function calls and declarations are case insensitive, leading and trailing whitespace is ignored, and whitespace in the middle is always treated as a single space. Don't forget to escape chars! Since 'strings' and {vars} designate inputs in a function call, always use \', \", \[, \], \{, \} when using those actual characters inside a function call's text, to prevent an input from forming: Clear userEC(\')s credentials MOD(-) Textual step's text COM(// not necessary for textual steps) Matching multiple declarations MOD(-) Matching multiple function declarations under the same parent FD(* F) A FD(* F) B F COM(// matches both line 2 and line 5) COM(// produces branches:) COM(// 1CLOSEP F, A) COM(// 2CLOSEP F, B) Making vars available below FD(* F) STR({name}) = STR('bob') COM(// this variable will be accessible after a function call to F) F Type STR({name}) into STR('textbox') COM(// will type 'bob') Making funcs available below FD(* F) FD(* A) B FD(* G) FD(* A) C FD(* A) D F COM(// makes public function A at line 2 available below) A COM(// B is run here) F COM(// makes public function A at line 2 available below) G COM(// makes public function A at line 6 available below) A COM(// C is run here) Equivalents COM(// file1.smash) COM(// -----------) FD(* A) FD(* B) MOD(-) 1 FD(* B) MOD(-) 2 COM(// file2.smash) COM(// -----------) FD(* A) FD(* B) MOD(-) 3 is equivalent to FD(* A) FD(* B) MOD(-) 1 MOD(-) 2 MOD(-) 3 Calls to itself FD(* Nav to homepage) Nav to STR('/') MOD(-) Some test Open Chrome Nav to homepage COM(// calls line 8) FD(* Nav to homepage) COM(// this "intercepts" navs to homepage to do security stuff) Do security checks Nav to homepage COM(// ignores line 8 because recursion not allowed, so calls line 1)

Variables Using COM(// In a function call) Buy tickets for STR({num adults}) adults and STR({{num children}}) children COM(// In a string literal) Nav to STR('{host}/path/name') COM(// {host} is replaced with its value) Setting {var} = 'str' STR({variable}) = STR('string') COM(// everything in Smashtest is a string) STR({variable}) = STR("string") STR({variable}) = STR([string]) STR({variable}) is STR('string') COM(// same as =) STR({variable})=STR('11'), STR({variable})=STR('22'), STR({variable})=STR('33') STR({variable})=STR('{host}/path/name') STR({variable})=STR('{var2}') COM(// cloned a var) STR({ Variable Name With Caps and Whitespace }) = STR('string') {var} = Func with code block STR({variable}) = Get hello COM(// variable is set to "hello world!") FD(* Get hello) { JSKEYWORD(return) STR("hello ") + STR("world!"); COM(// any kind of js value will work (not just stringCLOSEP) } STR({variable}) = Get goodbye { COM(// variable is set to "goodbye!") JSKEYWORD(return) STR("goodbye!"); } {var} = Func with branches FD(* A bad username) STR({x}) = STR('00') COM(// you can use any variable name, not just {x}) STR({x}) = STR('baduser') STR({x}) = STR('[none]') STR({x}) = STR('') STR({x}) = STR(' ') STR({username}) = A bad username Type STR({username}) into STR({textbox}) COM(// produces branches:) COM(// 1CLOSEP type '00') COM(// 2CLOSEP type 'baduser') COM(// 3CLOSEP type '[none]') COM(// 4CLOSEP type '') COM(// 5CLOSEP type ' ') Variable names are case sensitive (unlike step names), leading and trailing whitespace is ignored, and whitespace in the middle is always treated as a single space. They're case sensitive because they're converted to js vars in code blocks (and js is case-sensitive). Escape sequences You're not allowed to have a \ in a variable name, and you're not allowed to have a \0, \x, \u, or \c inside a 'string literal'. To get around this, use: STR({variable}) = special char string { JSKEYWORD(return) STR("\u2665 \cJ"); } Types {Global} Global variables are accessible to all steps for the rest of the branch (inside and outside function calls). STR({variable}) = STR('string') COM(// global variable) F FD(* F) Type STR({variable}) into STR('textbox') COM(// accessible here) F Type STR({variable}) into STR('textbox') COM(// accessible here) FD(* F) STR({variable}) = STR('string') {{Local}} Local variables are only accessible inside the current function call. STR({{variable}}) = STR('string') COM(// local variable) F Type STR({{variable}}) into STR('textbox') COM(// accessible here) FD(* F) COM(Type {{variable}} into 'textbox') COM(// NOT accessible here) F COM(Type {{variable}} into 'textbox') COM(// NOT accessible here) FD(* F) STR({{variable}}) = STR('string') Type STR({{variable}}) into STR('textbox') COM(// accessible here) COM(// goes out of scope here) Persistent Persistent variables exist for the lifetime of the whole suite run. They can only be get and set inside code blocks. They are usually used for internal stuff, like storing functions and libraries. Inside a code block Getting STR({variable}) = STR('something') Get a variable { JSKEYWORD(let) JSVARIABLE(v) = JSVARIABLE(variable); COM(// just use it as a js variable) COM(// for this to work, variable name must be a single word,) COM(// no chars other than A-Z, a-z, 0-9, - _ .) COM(// and not have the same name as a js keyword) COM(// when different types of vars have the same name,) COM(// local takes precedence over global, which takes precedence over persistent) } Get a local variable { JSKEYWORD(let) JSVARIABLE(v) = JSFUNC(l)(STR('variable name')); } Get a global variable { JSKEYWORD(let) JSVARIABLE(v) = JSFUNC(g)(STR('variable name')); } Get a persistent variable { JSKEYWORD(let) JSVARIABLE(v) = JSFUNC(p)(STR('variable name')); } Setting Set a local variable { JSFUNC(l)(STR('variable name'), STR('new value')); COM(// any kind of js value will work (not just stringCLOSEP) } Set a global variable { JSFUNC(g)(STR('variable name'), STR('new value')); } Set a persistent variable { JSFUNC(p)(STR('variable name'), STR('new value')); } Lookahead (:) STR({var)LC(:)STR(}) will get the value of the variable when it's set later in the branch. This allows you to refactor common steps higher up into the tree. Type STR({username)LC(:)STR(}) into STR('login box') COM(// ignores current value of {username}, looks to the first) COM(// {username}='str' line further in the branch) STR({username}) = STR('bob') STR({username}) = STR('mary') STR({username}) = STR('vishal') Verify success STR({username}) = STR('baduser') STR({username}) = STR('[none]') STR({username}) = STR('') Verify error Choose STR({adults)LC(:)STR(}) and STR({children)LC(:)STR(}) from reservations panel STR({adults})=STR('[none]') STR({adults})=STR('0') STR({children})=STR('[none]') STR({children})=STR('0') STR({children})=STR('1') STR({children})=STR('8') Verify error STR({adults})=STR('1') STR({adults})=STR('8') STR({children})=STR('[none]') STR({children})=STR('0') STR({children})=STR('1') STR({children})=STR('8') Verify success Setting a lookahead var's value You cannot set a lookahead var's value inside a code block, only in a STR({var})=STR('something') or a STR({var})=Function call (but the function call has to be sync - i.e., no awaits and an immediate return).

Code blocks Types FD(* Function name) { COM(// this is a code block) COM(// you can do anything js or nodejs supports) COM(// only lines between the "{" and "}" lines are part of the code block) } COM(// must end at the same indent level as the starting line) Step name { COM(// this step is implemented in here) COM(// kind of like a one-time function call) } Modifiers COM(// Modifiers can come before the name, or after the name but before the "{") MOD(..) MOD(+) FD(* Function name) MOD($) MOD(!) { } MOD(..) MOD(+) Step name MOD($) MOD(!) { } Await COM(// Code blocks are async, meaning you can use the 'await' keyword) Verify success { JSKEYWORD(await) JSFUNC($)(STR('#success')); } Don't forget await! Async js function calls should always be preceded with await. If errors are thrown but aren't captured inside a step, or if things just seem wonky, it's probably because you forgot an await. Prev COM(// Pass a value from one code block to the next via 'return' and 'prev') Step one { JSKEYWORD(return) STR("hello"); } Step two { JSFUNC(expect)(JSVARIABLE(prev)).JSVARIABLE(to).JSVARIABLE(equal)(STR("hello")); COM(// chai's expect and assert are available by default) } Step one { JSKEYWORD(return) STR("hello"); } Some intermediate step Step two { JSFUNC(expect)(JSVARIABLE(prev)).JSVARIABLE(to).JSVARIABLE(equal)(STR("hello")); } Variables Settings vars { JSFUNC(l)(STR('var1'), STR('new value')); COM(// local variable (any value will work, not just stringsCLOSEP) JSFUNC(g)(STR('var2'), STR('new value')); COM(// global variable) JSFUNC(p)(STR('var3'), STR('new value')); COM(// persistent variable) } Getting vars { JSKEYWORD(let) JSVARIABLE(v) = JSVARIABLE(var1); COM(// works if variable name is a single word with no special chars) JSKEYWORD(let) JSVARIABLE(v) = JSFUNC(l)(STR('var1')); COM(// local variable) JSKEYWORD(let) JSVARIABLE(v) = JSFUNC(g)(STR('var2')); COM(// global variable) JSKEYWORD(let) JSVARIABLE(v) = JSFUNC(p)(STR('var3')); COM(// persistent variable) } Use l()/g()/p() to set variables Don't set a {variable} with JSVARIABLE(variable) = STR('value'); as this will not persist past the end of the code block. Be mindful when using 'let' Don't use 'let' to declare a new variable with the same name as an existing {variable}, or you'll get a runtime error (double-declaration). You can use 'var' to get around this. Timeouts The default timeout for a step is 60 secs. If it hasn't completed by then, it will fail with a timeout error. Steps generally have their own, more stringent timeouts as well. These are usually implemented by an (async) function called within the step that has its own timeout. We made the default timeout really high since we wanted to leave the more "realistic" timeout to each step's individual discretion. The default timeout can be changed via JSFUNC(setStepTimeout)(secsCLOSEP for all steps after the current one in the current branch. Errors Normal error Failing step { JSKEYWORD(throw new) JSCLASS(Error)(STR("oops")); COM(// this step and branch will fail and end immediately) } Error.continue Failing step { JSKEYWORD(let) e = JSKEYWORD(new) JSCLASS(Error)(STR("verify failed, but let's try the next verify anyway")); JSVARIABLE(e).JSVARIABLE(continue) = JSCONST(true); COM(// this error will fail the step and branch,) JSKEYWORD(throw) JSVARIABLE(e); COM(// but the branch will continue running) } Error.fatal Failing step { JSKEYWORD(let) e = JSKEYWORD(new) JSCLASS(Error)(STR("something really bad")); JSVARIABLE(e).JSVARIABLE(fatal) = JSCONST(true); COM(// this error will end the whole test suite execution immediately) JSKEYWORD(throw) JSVARIABLE(e); } Stack traces A stack trace will contain something like this, where [LINE NUMBER] represents the line in the .smash file where the error occurred: at CodeBlock_for_[NAME OF CODE BLOCK FUNCTION] (eval at ...), <anonymous>:[LINE NUMBER]:[COL NUMBER] Implement complex functions in their own js files to generate more traditional stack traces: COM(// test.smash) COM(// ----------) Step { JSKEYWORD(const) JSVARIABLE(yf) = JSFUNC(i)(STR('yf'), STR('./yourfile.js')); JSVARIABLE(yf).JSFUNC(something)(); } COM(// yourfile.js) COM(// ----------) JSKEYWORD(function) JSFUNC(something)() { COM(// etc.) } JSVARIABLE(module.exports.something) = JSVARIABLE(something);

Code reference These js functions and objects are accessible within any code block. c() JSFUNC(c)(STR('print to console')); COM(// same as console.log(CLOSEP, but prints it out more neatly) COM(// (and clear of the progress barCLOSEP) dir() JSFUNC(dir)(); COM(// returns the absolute directory of the file where this step is) g() JSFUNC(g)(STR('variable'), STR('new value')); COM(// set global variable) JSFUNC(g)(STR('variable')); COM(// get global variable) getGlobal() JSFUNC(getGlobal)(STR('variable')); COM(// get global variable) getLocal() JSFUNC(getLocal)(STR('variable')); COM(// get local variable) getPersistent() JSFUNC(getPersistent)(STR('variable')); COM(// get persistent variable) i() JSKEYWORD(const) JSVARIABLE(packageName) = JSFUNC(i)(STR('package-name')); COM(// require(CLOSEP's (importsCLOSEP the given nodejs package,) COM(// sets persistent var 'packageName' to it (auto-camelCasedCLOSEP,) COM(// and returns it) JSKEYWORD(const) JSVARIABLE(myPkg) = JSFUNC(i)(STR('myPkg'), STR('package-name')); COM(// same, but sets the name of the persistent var) JSFUNC(i)(STR('myPkg'), STR('./path/to/file.js')); COM(// works with js files too) JSFUNC(i)(STR('myPkg'), STR('../path/to/file.js')); JSFUNC(i)(STR('myPkg'), STR('/Users/Shared/tests/file.js')); l() JSFUNC(l)(STR('variable'), STR('new value')); COM(// set local variable) JSFUNC(l)(STR('variable')); COM(// get local variable) log() JSFUNC(log)(STR('text to log')); COM(// logs a piece of text to the report for this step) p() JSFUNC(p)(STR('variable'), STR('new value')); COM(// set persistent variable) JSFUNC(p)(STR('variable')); COM(// get persistent variable) runInstance JSVARIABLE(runInstance); COM(// represents the test runner "thread" that's) COM(// running this step and branch (see RunInstance CLOSEP) JSVARIABLE(runInstance.runner); COM(// represents the test runner (see Runner CLOSEP) JSVARIABLE(runInstance.tree); COM(// represents the whole tree (see Tree CLOSEP) JSVARIABLE(runInstance.currBranch); COM(// represents the current branch (see Branch CLOSEP) JSVARIABLE(runInstance.currStep); COM(// represents the current step (see Step CLOSEP) COM(// These objects can be used to dynamically view, create, and/or edit tests at runtime) setGlobal() JSFUNC(setGlobal)(STR('variable'), STR('new value')); COM(// set global variable) setLocal() JSFUNC(setLocal)(STR('variable'), STR('new value')); COM(// set local variable) setPersistent() JSFUNC(setPersistent)(STR('variable'), STR('new value')); COM(// set persistent variable) setStepTimeout() JSFUNC(setStepTimeout)(JSCONST(30)); COM(// sets step timeout to 30 secs for all steps in this branch after this one)

Sequential (..) .. above a step block Simple case Nav to STR('/page') MOD(..) COM(// makes the whole step block run sequentially) Type STR('1111') into STR('textbox') Type STR('2222') into STR('textbox') Type STR('3333') into STR('textbox') Verify success COM(// produces 1 branch:) COM(// 1CLOSEP nav, type 1111, type 2222, type 3333, verify success) Function calls as step block members COM(// Note: Acts differently from function calls under a .. step.) COM(// If a function call has multiple branches, multiple branches will be generated:) FD(* Go to cart) COM(// two different ways of getting to the cart) Nav to STR('/cart') Click STR('cart icon') MOD(..) Go to cart Add peanuts to cart Verify peanuts added COM(// produces branches:) COM(// 1CLOSEP nav to /cart, add peanuts, verify peanuts) COM(// 2CLOSEP click cart icon, add peanuts, verify peanuts) .. on a step Simple case Sequential test MOD(..) COM(// flatten branches at or below me into one sequential branch) One Two Three Four Five COM(// produces 1 branch:) COM(// 1CLOSEP sequential test, one, two, three, four, five) With a step block Nav to STR('/page') MOD(..) Type STR('1111') into STR('textbox') Type STR('2222') into STR('textbox') Type STR('3333') into STR('textbox') Verify success COM(// produces 1 branch:) COM(// 1CLOSEP nav, type 1111, verify success, type 2222, verify success, type 3333, verify success) With a function call FD(* Type in) Type STR('1111') into STR('textbox') Type STR('2222') into STR('textbox') Type STR('3333') into STR('textbox') Nav to STR('/page') MOD(..) Type in Verify success COM(// produces 1 branch:) COM(// 1CLOSEP nav, type 1111, verify success, type 2222, verify success, type 3333, verify success) Nav to STR('/page') Type in MOD(..) COM(// all we did was move the .. down one line) Verify success COM(// produces the same branch as before) Inside a function declaration FD(* Open cart) MOD(..) COM(// the 3 steps here execute sequentially) Nav to STR('/') Click STR('cart icon') Verify STR('cart') is visible Open cart COM(// it's sequential inside, but not sequential out here) Do stuff Do more stuff COM(// produces 1 branch:) COM(// 1CLOSEP open cart (nav to /, click cart icon, verify cartCLOSEP, do stuff, do more stuff)

Non-parallel (!, !!) ! STR({username}) is STR('pete') MOD(!) COM(// no two branches going through this step may execute simultaneously) Nav to STR('/') COM(// useful for testing a stateful shared resource, like a test account) COM(// etc.) Nav to STR('/page1') COM(// etc.) !! STR({username}) is STR('bob') MOD(!!) COM(// no two branches going through this step may execute simultaneously,) Nav to STR('/') COM(// unless --test-server was set) COM(// etc.) COM(// useful for things like safaridriver, which can't run more) Nav to STR('/page1') COM(// than one instance locally, but can handle multiple instances) COM(// etc.) COM(// in a selenium grid)

Comments Standard use COM(// full-line comment) Step COM(// comment at the end of a step) Comments ignore whole lines Open Chrome COM(// this is still a valid step block) COM(// Open Firefox) COM(// if whole line starts with //, it's ignored as if it weren't there) Open Safari Do something COM(// produces branches:) COM(// 1CLOSEP open chrome, do something) COM(// 2CLOSEP open safari, do something) Inside code blocks Code block step { COM(// normal js comments occur here) COM(/* normal js comments occur here */) }

Groups and freq (#) Groups Open Chrome Open Firefox Navigate to STR('google.com') MOD(#google) MOD(#happy-path) Do a search Navigate to STR('pets.com') MOD(#pets) MOD(#happy-path) Buy cat food Only run Google branch: smashtest --groups=google Only run branches in happy path OR pets: smashtest --groups="happy-path,pets" Only run branches in happy path AND pets: smashtest --groups="happy-path+pets" Only run Firefox branches: smashtest --groups=firefox (built-in group) Only run branches in happy path AND pets, or firefox AND pets: smashtest --groups="happy-path+pets,firefox+pets" Frequency Open Chrome Navigate to STR('google.com') Do something you want tested very often MOD(#high) COM(// good for quick smoke tests) Do something you want usually tested MOD(#med) COM(// your normal test suite) Do something you want usually tested COM(// #med is the default freq of a branch if #high/med/low is omitted) Do something you want tested once in a while MOD(#low) COM(// good for long-running, low-risk, edge-casey stuff) This branch will be med MOD(#med) MOD(#some-group) COM(// the later step controls the branch's freq) This branch will be low Run low/med/high tests: smashtest --min-frequency=low Run med/high tests: smashtest --min-frequency=med Run high tests: smashtest --min-frequency=high Branches are also run in order of frequency, from high to low (and are shuffled within their freq).



MOD(#desktop) , MOD(#mobile) ,

MOD(#chrome) , MOD(#firefox) , MOD(#safari) , MOD(#ie) , MOD(#edge) ,

MOD(#low) , MOD(#med) , MOD(#high) These are the reserved hashtags used by smashtest

Conditional tests It's best to avoid conditional tests. They should only be used to implement rare exceptions. If step If A then B { JSKEYWORD(if)(JSVARIABLE(A)) { COM(// only does B if A is true) JSFUNC(B)(); } } If browser is... MOD(-) Test something Do not allow Safari { JSKEYWORD(if)(JSVARIABLE(browser.params.name) == STR('safari')) { COM(// pass and end the whole branch if the browser is safari, do nothing otherwise) JSVARIABLE(runInstance.currBranch).JSFUNC(markBranch)(STR('pass')); } } MOD(-) This step only runs if the browser isn't safari If viewport is... MOD(-) Test something If mobile { JSKEYWORD(if)(EC(!)JSVARIABLE(runInstance.currBranch.groups).JSFUNC(includes)(STR('mobile'))) { COM(// pass and end the whole branch if the viewport isn't mobile, do nothing otherwise) JSVARIABLE(runInstance.currBranch).JSFUNC(markBranch)(STR('pass')); } } MOD(-) This step only runs if the viewport is mobile

Skipping (-, -s, .s, $s) Skip one step -s (recommended) One COM(// runs) Skipped step MOD(-s) COM(// doesn't run, marked skipped in report) Two COM(// runs) - One COM(// runs) Skipped step MOD(-) COM(// doesn't run, regular textual step in report) Two COM(// runs) // One COM(// runs) COM(// Skipped step) COM(// doesn't run, not outputted to report) Two COM(// will run, but needs to be dedented once) Skip all branches passing through a step $s One COM(// doesn't run) Two COM(// doesn't run) Skipped step MOD($s) COM(// skips any branch that passes through this step, still expands function calls (error if declaration not foundCLOSEP) Three COM(// doesn't run) Four COM(// doesn't run) Skip step and all steps below .s One COM(// runs) Two COM(// runs) Skipped step MOD(.s) COM(// doesn't run, still expands function calls (error if declaration not foundCLOSEP) Three COM(// doesn't run) Four COM(// doesn't run) COM(// Also skips entire duplicate branches caused by .s) COM(// Be careful, when .s is inside a function declaration it will skip steps after the function call as well) // One COM(// runs) Two COM(// runs) COM(// Skipped step) COM(// doesn't run) COM(//) COM(// Three) COM(// doesn't run) COM(// Four) COM(// doesn't run) COM(// Useful if you don't want function calls to expand,) COM(// but won't remind you in the console/report that skipped steps exist) // on step block member One COM(// runs) COM(// Two) COM(// doesn't run, and doesn't run any steps below) Three COM(// runs) Four COM(// runs, except for "Two")

Collapsing (+, +?) Collapse (+) FD(* Select best item from dropdown) MOD(+) COM(// function calls here will be collapsed by default in the report) Click STR('dropdown') Scroll to STR('best item') Click STR('best item') Some big operation with lots of steps MOD(+) COM(// this function call will be collapsed by default in the report) If there's an error inside a MOD(+)'ed function, or if the function is currently running, it will be uncollapsed automatically. It's recommended to mark precondition steps with MOD(+), since their details aren't central to the test. Hidden (+?) FD(* Init) MOD(+?) COM(// function calls here will be hidden in the report) Do some internal stuff COM(// this function call will be hidden in the report) Some internal thing somebody reading the report doesn't care about MOD(+?) If there's an error inside a MOD(+?)'ed function, it will be visible in the report.

Only ($) One $ Open Chrome Navigate to STR('google.com') Do a search DEBUG($ Navigate to 'pets.com') COM(// only runs branches that pass through this step) Buy cat food COM(// i.e., the branch ending here) Multiple $'s at different indents Open Chrome DEBUG($ Open Firefox) Open Safari Desktop DEBUG($ Mobile) Navigate to STR('google.com') Do a search COM(// only runs the Firefox mobile branch that ends here) Multiple $'s with the same parent Open Chrome DEBUG($ Open Firefox) DEBUG($ Open Safari) Desktop Mobile Navigate to STR('google.com') Do a search COM(// only runs the Firefox and Safari branches (4 of themCLOSEP) On a function declaration DEBUG($ * Do a search) COM(// only runs branches that call this function) COM(// etc.) Log in as STR('bob') Do a search Log in as STR('vishal') Do a search With ~'s Open Chrome DEBUG(Open Firefox $) Open Safari Desktop DEBUG(Mobile $) DEBUG(Navigate to 'google.com' ~) COM(// use to help isolate a branch for a debug)

Debug (~, ~~) Debug modifier (~) Navigate to STR('google.com') MOD(~ Click 'button') COM(// isolate this branch, run in REPL, and pause right before this step) Click STR('other thing') Navigate to STR('google.com') MOD(Click 'button' ~) COM(// isolate this branch, run in REPL, and pause right after this step) Click STR('other thing') A branch run in debug will also pause after a failing step. If a MOD(~) goes through multiple branches, the first one is chosen. No report is generated and the list of previously passed branches (for -s) is retained. Express debug modifier (~~) Navigate to STR('google.com') MOD(~~ Click 'button') COM(// only run this branch, but no pausing and don't run in REPL) Click STR('other thing') Just like for MOD(~), no report is generated and the list of previously passed branches (for -s) is retained. Debug flag You can also run smashtest --debug=[hash] to debug the branch with the given hash. Be careful. If you change any step, the hash of its branch will change. The recommended technique is to place a MOD($), MOD(~~), or MOD(~) on a line you change, then run that. You'll want to rerun all the branches that pass through that line anyway.

Hooks (***) What's a hook? A hook is a code block function that runs before or after a step, branch, or test suite. They're not meant for testing or setup/teardown. They're for internal stuff, such as reporting, importing code, js function declarations, screenshots, and logging. Only code blocks are allowed, and they cannot have children or modifiers. Hooks aren't listed in the report. If a hook fails, the step/branch it corresponds to will take the error and fail. Types Before Every Branch Parent step A B FD(*** Before Every Branch) { COM(// this code runs before every branch that goes through the parent step) COM(// i.e.,) COM(// 1CLOSEP this code, parent step, A) COM(// 2CLOSEP this code, parent step, B) } FD(*** Before Every Branch) { COM(// this code runs before every branch in existence) } After Every Branch Parent step FD(*** After Every Branch) { COM(// this code runs after every branch that goes through the parent step (whether it passes or failsCLOSEP) } FD(*** After Every Branch) { COM(// this code runs after every branch in existence (whether it passes or failsCLOSEP) } Before Every Step Parent step FD(*** Before Every Step) { COM(// this code runs before every step of every branch that goes through the parent step) } FD(*** Before Every Step) { COM(// this code runs before every step of every branch in existence) } After Every Step Parent step FD(*** After Every Step) { COM(// this code runs after every step of every branch that goes through the parent step) } FD(*** After Every Step) { COM(// this code runs after every step of every branch in existence) } Before Everything FD(*** Before Everything) { COM(// this code runs before the whole test suite begins) COM(// only valid at 0 indents) } After Everything FD(*** After Everything) { COM(// this code runs after the whole test suite ends) COM(// only valid at 0 indents) } Where should setup and teardown logic go? Setup code should go in the actual test. Teardown code should go in the same setup logic, such that the previous state is cleaned out prior to actual testing. It should go into a hook only if necessary. MOD(-) My test Setup Testing FD(* Setup) Clean previous state Set things up for new test

UI Testing Smashtest is a general platform for testing which comes with several built-in packages. One of these packages handles web UI testing with selenium webdriver. This section discusses the steps and js functions that this package makes available. Also, check out this UI test example.

Browsers and devices Open a browser All UI steps MUST come after an "Open [browser]" step Open Chrome COM(// run exclusively with --groups=chrome) Open Firefox COM(// run exclusively with --groups=firefox) Open Safari COM(// run exclusively with --groups=safari) Open IE COM(// run exclusively with --groups=ie) Open Edge COM(// run exclusively with --groups=edge) Open browser STR('chrome') COM(// use a string recognized by webdriver as a browser name) Set viewport size COM(// Run these exclusively with --groups=desktop) Desktop COM(// sets browser to 1920 x 1080) Laptop COM(// sets browser to 1024 x 768) Laptop L COM(// sets browser to 1440 x 900) COM(// Run these exclusively with --groups=mobile) Mobile COM(// sets browser to 375 x 667) Mobile Portrait COM(// sets browser to 375 x 667) Mobile Landscape COM(// sets browser to 667 x 375) Mobile S COM(// sets browser to 320 x 480) Mobile S Portrait COM(// sets browser to 320 x 480) Mobile S Landscape COM(// sets browser to 480 x 320) Mobile M COM(// sets browser to 375 x 667) Mobile M Portrait COM(// sets browser to 375 x 667) Mobile M Landscape COM(// sets browser to 667 x 375) Mobile L COM(// sets browser to 425 x 667) Mobile L Portrait COM(// sets browser to 425 x 667) Mobile L Landscape COM(// sets browser to 667 x 425) Tablet COM(// sets browser to 768 x 1024) Tablet Portrait COM(// sets browser to 768 x 1024) Tablet Landscape COM(// sets browser to 1024 x 768) Set device type COM(// These steps set the viewport to the given device, and do device emulation in Chrome) BlackBerry Z30 COM(// sets browser to 360 x 640) Blackberry PlayBook COM(// sets browser to 600 x 1024) Galaxy Note 3 COM(// sets browser to 360 x 640) Galaxy Note II COM(// sets browser to 360 x 640) Galaxy S III COM(// sets browser to 360 x 640) Galaxy S5 COM(// sets browser to 360 x 640) Kindle Fire HDX COM(// sets browser to 800 x 1280) LG Optimus L70 COM(// sets browser to 384 x 640) Laptop with HiDPI screen COM(// sets browser to 1440 x 900) Laptop with MDPI screen COM(// sets browser to 1280 x 800) Laptop with touch COM(// sets browser to 1280 x 950) Microsoft Lumia 550 COM(// sets browser to 640 x 360) Microsoft Lumia 950 COM(// sets browser to 360 x 640) Nexus 4 COM(// sets browser to 384 x 640) Nexus 5 COM(// sets browser to 360 x 640) Nexus 5X COM(// sets browser to 412 x 732) Nexus 6 COM(// sets browser to 412 x 732) Nexus 6P COM(// sets browser to 412 x 732) Nexus 7 COM(// sets browser to 600 x 960) Nexus 10 COM(// sets browser to 800 x 1280) Nokia Lumia 520 COM(// sets browser to 320 x 533) Nokia N9 COM(// sets browser to 480 x 854) Pixel 2 COM(// sets browser to 411 x 731) Pixel 2 XL COM(// sets browser to 411 x 823) iPhone 4 COM(// sets browser to 320 x 480) iPhone 5/SE COM(// sets browser to 320 x 568) iPhone 6/7/8 COM(// sets browser to 375 x 667) iPhone 6/7/8 Plus COM(// sets browser to 414 x 736) iPhone X COM(// sets browser to 375 x 812) iPad COM(// sets browser to 768 x 1024) iPad Mini COM(// sets browser to 768 x 1024) iPad Pro COM(// sets browser to 1024 x 1366) Usage examples Open Chrome Open Firefox Open Safari Desktop MOD(-) Desktop test 1 COM(// etc.) MOD(-) Desktop test 2 COM(// etc.) Mobile MOD(-) Mobile test 1 COM(// etc.) MOD(-) Mobile test 2 COM(// etc.) iPhone X MOD(-) iPhone X test COM(// Login function with different steps on desktop vs. mobile) COM(// Run with `smashtest` or `smashtest *.smash`) COM(// ----- main.smash -----) Open Chrome On Desktop On Mobile COM(// etc.) Login COM(// the login that's called depends on desktop vs. mobile) COM(// ----- viewports.smash -----) FD(* On Desktop) Desktop COM(// calls `Desktop` step exposed by `Open Chrome`) FD(* On Mobile) Mobile COM(// calls `Mobile` step exposed by `Open Chrome`) COM(// ----- desktop.smash -----) FD(* On Desktop) FD(* Login) MOD(-) Desktop implementation of Login COM(// ----- mobile.smash -----) FD(* On Mobile) FD(* Login) MOD(-) Mobile implementation of Login

Capabilities Sometimes you need to set custom browser capabilities and options, such as when you need to provide a username and password for a cloud service (BrowserStack, Sauce Labs, etc.) Set custom capabilities Open Chrome Set custom capabilities { JSFUNC(g)(STR('browser capabilities'), { STR('name'): STR('foobar') COM(// This is the Capabilities object. Capabilities go here.) COM(// See withCapabilities(CLOSEP ) }); } COM(// Note: you can set this before or after the "Open [browser]" step) Set custom options Open Chrome Set custom options { JSKEYWORD(const) JSVARIABLE(chrome) = JSFUNC(i)(STR('selenium-webdriver/chrome')); JSKEYWORD(let) JSVARIABLE(opts) = JSKEYWORD(new) JSVARIABLE(chrome).JSFUNC(Options)(); COM(// Call functions on opts here) COM(// See set[browser]Options(CLOSEP functions ) JSFUNC(g)(STR('browser options'), opts); } COM(// Note: you can set this before or after the "Open [browser]" step)

Browser steps You must run an Open [browser] step before using any of the steps below. {{element}} below can either be an ElementFinder string or an existing WebElement object. All steps that involve mouse interaction (i.e., clicking) choose the first clickable element that matches {{element}}. If nothing found, they choose the first non-clickable element that matches {{element}}. Interact Click STR('elementfinder') Native click STR('elementfinder') COM(// same as click, but uses js click instead of webdriver click) COM(// try this when webdriver click doesn't work) Double click STR('elementfinder') Hover over STR('elementfinder') Scroll to STR('elementfinder') Check STR('elementfinder') COM(// clicks the element, if it's currently unchecked) Uncheck STR('elementfinder') COM(// clicks the element, if it's currently checked) Set Type STR('text') into STR('elementfinder') Type STR('text[enter]') into STR('elementfinder') COM(// see list of keys (case-insensitiveCLOSEP) Type STR('[none]') into STR('elementfinder') COM(// step does nothing) COM(// good for including inaction when testing different inputs) Clear STR('elementfinder') COM(// clear a textbox, etc.) Set STR('elementfinder') to STR('value') Set STR('elementfinder') to STR('[none]') COM(// step does nothing) COM(// good for including inaction when testing different inputs) Select STR('6') from STR('elementfinder') COM(// selects an from a ) COM(// if an with this exact value cannot be found,) COM(// searches for an that contains the value,) COM(// trimmed and case-insensitive) Select STR('[none]') from STR('elementfinder') COM(// step does nothing) COM(// good for including inaction when testing different inputs) Select element STR('option elementfinder') from STR('dropdown elementfinder') COM(// selects an from a ) STR({variable}) = Value of STR('elementfinder') Navigate Navigate to STR('https://site.com/page') Navigate to STR('http://site.com/page') Navigate to STR('site.com/page') COM(// defaults to http) Navigate to STR('/page') COM(// uses domain browser is currently on) Nav to STR('/page') COM(// shorthand for Navigate) Go Back Go Forward Refresh STR({current url}) = Current url COM(// returns current absolute url) Window Set dimensions to width=STR('1024') height=STR('768') Maximize window Open new tab COM(// opens new tab ("window"CLOSEP and switches to it) Switch to window whose title contains STR('hello') Switch to window whose url contains STR('/page') Switch to the STR('1st') window Switch to the STR('4th') window Switch to iframe STR('elementfinder') Switch to topmost iframe STR({window title}) = Window title COM(// returns current window title) Alerts Accept alert COM(// clicks ok in alert modal, error if no alert open) Dismiss alert COM(// clicks cancel in alert modal, error if no alert open) Wait Avoid this kind of wait. Instead, consider Verify or Wait Until Wait STR('2') secs COM(// sleeps this long) Wait STR('2') seconds Wait STR('1') sec Wait STR('1') second Cookies and storage STR({cookie}) = Get cookie STR('name') Verify cookie { JSVARIABLE(cookie); COM(// object containing cookie info) JSVARIABLE(cookie.value); COM(// value of cookie) } Set cookie STR('name') to STR('value') Set cookie STR('name') to STR('value'), expiring in STR('65') secs Delete cookie STR('name') Delete all cookies Clear local storage Clear cookies and local storage Print and log Log STR('text to log to this step in report') STR('elementfinder') COM(// outputs found elements to browser's console) STR("elementfinder") COM(// and number of found elements to regular console) STR([elementfinder]) COM(// useful when using REPL)

Verify steps You must run an Open [browser] step before using any of the steps below. Verify 'Verify' steps wait up to 2 secs for the verify to pass before failing. Verify STR('elementfinder') is visible Verify STR('elementfinder') is not visible Verify STR('elementfinder') is STR('state-elementfinder') Verify every STR('elementfinder') is STR('state-elementfinder') Verify at page STR('Page title') COM(// passes if current page title (case-insensitiveCLOSEP) Verify at page STR('site.com/page') COM(// or url contains this text) Verify at page STR('/page') Verify at page STR('Page .*') COM(// passes if current page title) Verify at page STR('(.*CLOSEPpage') COM(// or url matches this regex) Verify at page STR('^http(.*CLOSEPpage.+$') Verify cookie STR('name') contains STR('value') Verify alert contains STR('hello') COM(// passes if alert is open and contains this text) COM(// Note: this step doesn't wait up to 2 secs - it's immediate) Wait until 'Wait until' steps wait up to 15 secs, or the specified amount, for the verify to pass before failing. Wait until STR('elementfinder') is visible Wait until STR('elementfinder') is visible (up to STR('30') secs) Wait until STR('elementfinder') is not visible Wait until STR('elementfinder') is not visible (up to STR('30') secs) Wait until STR('elementfinder') is STR('state-elementfinder') Wait until STR('elementfinder') is STR('state-elementfinder') (up to STR('30') secs) Wait until every STR('elementfinder') is STR('state-elementfinder') Wait until every STR('elementfinder') is STR('state-elementfinder') (up to STR('30') secs) Wait until at page STR('Page title') COM(// passes if current page title (case-insensitiveCLOSEP) Wait until at page STR('site.com/page') COM(// or url contains this text) Wait until at page STR('/page') Wait until at page STR('Page .*') COM(// passes if current page title) Wait until at page STR('(.*CLOSEPpage') COM(// or url matches this regex) Wait until at page STR('^http(.*CLOSEPpage.+$') Wait until at page STR('Page title') (up to STR('30') secs) Wait until at page STR('site.com/page') (up to STR('30') secs) Wait until at page STR('/page') (up to STR('30') secs) Wait until at page STR('Page .*') (up to STR('30') secs) Wait until at page STR('(.*CLOSEPpage') (up to STR('30') secs) Wait until at page STR('^http(.*CLOSEPpage.+$') (up to STR('30') secs) Wait until cookie STR('name') contains STR('value') Wait until cookie STR('name') contains STR('value') (up to STR('30') secs) Assert Verify STR({variable}) equals STR('value') Verify STR({variable}) is STR('value') Verify STR({variable}) == STR('value') Verify STR({variable}) is greater than STR('value') Verify STR({variable}) > STR('value') Verify STR({variable}) is greater than or equal to STR('value') Verify STR({variable}) >= STR('value') Verify STR({variable}) is less than STR('value') Verify STR({variable}) < STR('value') Verify STR({variable}) is less than or equal to STR('value') Verify STR({variable}) <= STR('value')

Network conditions and throttling Chrome only Since Chrome is the only browser that supports emulating network conditions, this step only works in that browser. In all other browsers, this step quietly does nothing. See conditional tests for ways of excluding other browsers, if it's cleaner that way. You must run an Open [browser] step before using the step below. COM(// Makes the browser emulate the given network conditions) COM(// latency is additional latency in ms) COM(// max download and upload speeds are in bytes/sec) Set network conditions to offline=STR('true') latency=STR('200') max-download-speed=STR('300000') max-upload-speed=STR('400000')

Mocking time and geolocation You must run an Open [browser] step before using any of the steps below. Time COM(// Makes the browser think the date and time is the one that's given (hijacks js DateCLOSEP) COM(// Takes any string the js Date object can interpret) Mock time to STR('4/1/2003') Mock time to STR('2011 Aug 19 4:45 pm') Mock time to STR('2020-09-02 19:19:45') COM(// Where {date} contains a js Date object) Mock time to STR({date}) Geolocation COM(// Makes the browser think the user's current location is the one that's given) Mock location to latitude=STR('28.538336') longitude=STR('-81.379234') COM(// that's Orlando, FL, USA) COM(// Current pre-defined locations (case-insensitiveCLOSEP) Mock location to STR('Berlin') Mock location to STR('London') Mock location to STR('Moscow') Mock location to STR('New York') Mock location to STR('Mumbai') Mock location to STR('San Francisco') Mock location to STR('Seattle') Mock location to STR('Shanghai') Mock location to STR('São Paulo') Mock location to STR('Sao Paulo') Mock location to STR('Tokyo') Stop COM(// Stops all time, geolocation, and http mocks and restores originals) Stop all mocks

Mocking APIs You must run an Open [browser] step before using any of the js functions listed below. The following js functions work with both GET and POST String response Mock an endpoint { JSKEYWORD(await) JSFUNC(mockHttp)(STR('GET'), STR('/endpoint'), STR('canned response')); } COM(// An XHR GET from the browser to /endpoint will always get a) COM(// 200 with 'canned response' body) JSON response Mock an endpoint with a JSON response { JSKEYWORD(await) JSFUNC(mockHttp)(STR('GET'), STR('/endpoint'), {key: STR('val')}); } COM(// An XHR GET from the browser to /endpoint will always get a) COM(// 200 with json body '{"key":"val"}') Detailed response Mock an endpoint and specify the response status code, http headers, and body { JSKEYWORD(await) JSFUNC(mockHttp)(STR('GET'), STR('/endpoint'), [JSCONST(201), {STR('Content-Type'): STR('text/plain')}, STR('canned response')] ); } COM(// An XHR GET from the browser to /endpoint will always get a) COM(// 201 with the given http headers and 'canned response' body) Function response Mock an endpoint with a function { JSKEYWORD(await) JSFUNC(mockHttp)(STR('GET'), STR('/endpoint'), JSKEYWORD(function)(xhr) { JSVARIABLE(xhr).JSFUNC(respond)(JSCONST(201), {STR('Content-Type'): STR('text/plain')}, STR('canned response')); }); } COM(// An XHR GET from the browser to /endpoint will always get a) COM(// 201 with the given http headers and 'canned response' body) Regex endpoint Mock every endpoint that matches a regex { JSKEYWORD(await) JSFUNC(mockHttp)(STR('GET'), STR(/)EC(\/end.*)STR(/), STR('canned response')); } COM(// An XHR GET from the browser to any matching endpoint will always get a) COM(// 200 with 'canned response' body) Stop mocks Stop all http mocks and restore original endpoints { JSKEYWORD(await) JSFUNC(mockHttpStop)(); } Configure Configure the mock server { JSKEYWORD(await) JSFUNC(mockHttpConfigure)({autoRespond: JSCONST(false)}); COM(// See "Fake server options" at sinon's page for a list of configuration options) } Check out sinon's fake xhr and server (the underlying library) for more details on what you can do.

ElementFinders What's an ElementFinder? An ElementFinder (or EF) is a string that matches elements on a page. They contain one or more lines, where each line is a comma-separated list of props. A prop (or property) describes an element, or the state of an element. Click STR('login button') COM(// login button is an EF consisting of one prop, login button ) Click STR(['follow', next to 'bob']) COM(// 'follow', next to 'bob' is an EF) COM(// 'follow' and next to 'bob' are both props) COM(// remember that [brackets] delimit strings, like "quotes" or 'quotes') Alternatively, a list of props may be separated by spaces, but only if each item is either 'text', an ord (see below), or a prop that's already been defined (multiple words are ok, but no 'inputs'). Click STR(['Login' button]) COM(// 'Login' button is an EF, 'Login' and button are both props) Click STR([4th 'Login' button]) COM(// 4th (the ord propCLOSEP, must come first in space-separated EFs) Click STR('red button') COM(// if red button is not already defined, it will try to) COM(// interpret this as a list of two props, red and button ) Use space-separated inside steps You should use space-separated EFs in steps (such as Click) because they sound natural and are easier to read. For example, Click STR([red 'login' button]) sounds better than Click STR([button, red, 'login']). The JSFUNC($)() function is used to find an element given an EF. Since it throws an error if nothing is found in time, JSFUNC($)() can also be used to verify the existence (and visibility) of an element. login box ) JSKEYWORD(await) JSFUNC($)(STR(`login box`)); } Get login box { COM(// Sets elem to the first visible login box ) JSKEYWORD(let) JSVARIABLE(elem) = JSKEYWORD(await) JSFUNC($)(STR(`login box`)); } Verify focused login box { COM(// Verifies the existence of at least one visible element) COM(// that matches the EF with these 4 props) JSKEYWORD(await) JSFUNC($)(STR(`login box, focused, 'text inside', .selector`)); COM(// Similar example using props separated by either spaces or commas) COM(// css selectors and props with input have to be separated with commas) JSKEYWORD(await) JSFUNC($)(STR(`focused 'text inside' login box, .selector`)); } COM(// See Verify login box { COM(// Verifies the existence of at least one visible element) COM(// that matches the EF with prop) JSKEYWORD(await) JSFUNC($)(STR(`login box`)); } Get login box { COM(// Sets elem to the first visible WebElement ) COM(// that matches the EF with prop) JSKEYWORD(let) JSVARIABLE(elem) = JSKEYWORD(await) JSFUNC($)(STR(`login box`)); } Verify focused login box { COM(// Verifies the existence of at least one visible element) COM(// that matches the EF with these 4 props) JSKEYWORD(await) JSFUNC($)(STR(`login box, focused, 'text inside', .selector`)); COM(// Similar example using props separated by either spaces or commas) COM(// css selectors and props with input have to be separated with commas) JSKEYWORD(await) JSFUNC($)(STR(`focused 'text inside' login box, .selector`)); } COM(// See code reference for details on $(CLOSEP) Likewise, JSFUNC($$)() finds multiple matching elements, given an EF. To play around with EFs, run smashtest -r, open a browser, nav to a page, and type in various quoted STR('EFs') (or, you can run a test in debug). The browser console (DevTools) will contain logs for every EF match. Props Use defined props whenever possible Although selectors can be valid props on their own, avoid steps like Click STR('#some-elem'). Instead, define a prop and use it in the step, e.g., Click STR('login box'). Your tests will be easier to read and refactorable. Setting Props are defined with the JSFUNC(props)() function. More on that on the code reference page. On homepage { COM(// Define props for all the elements on the homepage) COM(// Format: 'prop name': `EF` or function) JSFUNC(props)({ STR('login box'): STR(`.msgbox, enabled, 2nd`), STR('about link'): STR(`selector "a[name='about']"`) }); } Matching rules Starting with all elements in the DOM, as props are applied (left to right), each one narrows down the list of elements. The elements that remain at the end are the ones that match the EF. Whenever a prop is encountered, it is interpreted according to the first rule it matches in the following list: 'text' Matches elements where the given text is contained in its innerText, value, placeholder, associated label's innerText, or currently selected <option>'s innerText (for selects).

Case insensitive, leading and trailing whitespace is ignored, and all whitespace is treated as a single space (both the 'text' and text in the DOM).

1st, 2nd, 3rd, etc. (ord) E.g., 4th = take the elements currently matched and choose only the 4th one.

In a comma-separated prop list, you will usually list an ord last, since it narrows you down to just one element.

In a space-separated prop list, the ord must come first (it's more natural-sounding that way), but will be applied last.

defined prop These are the definitions set by JSFUNC(props)()

They may take an input string, such as STR(propname 'input string') You must use commas to separate these from other props

There are several props that come pre-defined

css selector If nothing else matches, a prop is interpreted to be a css selector

You must use commas to separate these from other props If an EF fails to match an element, it will throw an error containing the full EF, prettified, with a --> explanation next to the lines that didn't match. Mind your selectors! If you use a tagname selector and it actually matches the name of a defined prop, the defined prop will be used. To be safe, use STR(selector 'tagname') as your prop. The same is true for selectors containing commas (ORs). Always use STR(selector '.selector1, .selector2') since commas separate props by default. Implicit visible prop By default, all EFs get an implicit STR(visible) prop (meaning, only match visible elements). The exception is when you explicitly use a STR(visible), STR(not visible), or STR(any visibility) prop. More on those here. Not Start a prop with "not" to find elements that don't match that prop. E.g., STR(not fuzzy) or STR(not .selector), Counters Match multiple elements by preceding a line with a counter. JSKEYWORD(await) JSFUNC($$)(STR(`login box`)); COM(// match 1 or more login boxes) JSKEYWORD(await) JSFUNC($$)(STR(`1 x login box`)); COM(// match exactly 1 login box) JSKEYWORD(await) JSFUNC($$)(STR(`3 x login box`)); COM(// match exactly 3 login boxes) JSKEYWORD(await) JSFUNC($$)(STR(`0+ x login box`)); COM(// match 0 or more login boxes) JSKEYWORD(await) JSFUNC($$)(STR(`1+ x login box`)); COM(// match 1 or more login boxes) JSKEYWORD(await) JSFUNC($$)(STR(`2+ x login box`)); COM(// match 2 or more login boxes) JSKEYWORD(await) JSFUNC($$)(STR(`2- x login box`)); COM(// match 2 or more login boxes) JSKEYWORD(await) JSFUNC($$)(STR(`2-5 x login box`)); COM(// match between 2 and 5 login boxes, inclusive) Child elements Simple example COM(// Matches one or more .list elements that contain these 3 children, in that order:) JSKEYWORD(await) JSFUNC($$)(STR(`) STR( .list) STR( .item1) STR( .item2) STR( .item3) STR(`)); COM(// This is a subset matching, meaning that other elements) COM(// can exist in and around the 3 children inside .list) COM(// The top parent (.listCLOSEP can start at any indentation that's a multiple of 4) Multi-level with counters JSKEYWORD(await) JSFUNC($$)(STR(`) STR( 1+ x .list) COM(// matches 1 or more .list elements that contain these children:) STR( 4 x .item) COM(// 4 .item's) STR( button[name=q]) COM(// 1 button with attribute name set to "q") COM(// blank lines are ok) STR( .section) COM(// 1 .section that contains these children:) STR( #textbox) COM(// 1 #textbox) STR( enabled login box) COM(// 1 login box that's enabled) STR( button, contains 'click me') COM(// 1 button that contains 'click me') STR(`)); COM(// Note that // comments are allowed inside EFs) Any order JSKEYWORD(await) JSFUNC($$)(STR(`) STR( #list) STR( any order) COM(// the 3 children can be in any order) STR( .item, "NYC") STR( .item, "Tokyo") STR( .item, "Paris") STR(`)); Matching children COM(// A [] around a line will match and return those elements,) COM(// as opposed to the top parent) COM(// This EF will match 4 elements:) COM(// a 'Tokyo' item and the 3 items that follow) JSKEYWORD(await) JSFUNC($$)(STR(`) STR( #list) STR( .item, 'NYC') STR( [.item, 'Tokyo']) STR( [3 x .item]) STR(`)); Mind your []'s! To match the selector STR([attr="something"]), use the prop STR(selector '[attr="something"]'). Otherwise, the []'s will be interpreted as a matching operator. Implicit body JSKEYWORD(await) JSFUNC($$)(STR(`) STR( #one) COM(// multiple lines at the top indent) STR( #two) STR(`)); COM(// implicitly equates to) JSKEYWORD(await) JSFUNC($$)(STR(`) STR( body) STR( #one) STR( #two) STR(`)); Element array Do stricter validations with element arrays: COM(// An element array is a line that starts with a *) COM(// It will match as many elements as it can, and verify that) COM(// all children listed underneath map 1-to-1 with each element matched.) COM(// The ordering and number must be the same. If not, an error occurs.) JSKEYWORD(await) JSFUNC($$)(STR(`) STR( #list) STR( * .item) COM(// this is an element array) STR( .item, 'NYC') STR( .item, 'Tokyo') STR( 2 x .item) STR(`)); COM(// There must be exactly 4 items inside of #list:) COM(// a 'NYC', a 'Tokyo', and 2 more items of any kind) COM(// in that exact order. Otherwise you get an error.) COM(// Note: You can include the `any order` keyword, as shown before,) COM(// to allow any ordering of the children)

Default ElementFinder props Smashtest defines these props by default. Remember, you can put STR(not) in front of any of them. STR(visible) = matches if an element is visible to the user (width and height > 0, no hidden styles, opacity > 0, opacity of all ancestors > 0) This prop is implicitly applied to all EFs, unless 'any visibility', 'visible', or 'not visible' are already in that EF

= matches if an element is visible to the user (width and height > 0, no hidden styles, opacity > 0, opacity of all ancestors > 0) STR(any visibility) = matches elements regardless of visibility, disables implicit 'visible' prop If you define a new prop as STR('new prop'): STR(`#some-invisible-elem, any visibility`) , you must use that new prop as Click STR('new prop, any visibility') Likewise, if you define another prop as STR('even newer prop'): STR(`new prop, any visibility`) , you must use that prop as Click STR('even newer prop, any visibility') , applying 'any visibility' all the way up the chain

= matches elements regardless of visibility, disables implicit 'visible' prop STR(enabled) = matches if 'disabled' attribute isn't present

= matches if 'disabled' attribute isn't present STR(disabled) = matches if 'disabled' attribute is present

= matches if 'disabled' attribute is present STR(checked) = matches if element.checked is true

= matches if element.checked is true STR(unchecked) = matches if element.checked is false

= matches if element.checked is false STR(selected) = matches if element.selected is true (used for selects)

= matches if element.selected is true (used for selects) STR(focused) = matches if element currently has focus

= matches if element currently has focus STR(element) = matches any element

= matches any element STR(clickable) = matches if element is of a clickable type (a, button, label, input, textarea, select, option) or has cursor:pointer style

= matches if element is of a clickable type (a, button, label, input, textarea, select, option) or has cursor:pointer style STR(page title 'title') = causes error if the page title isn't equal to the given string

= causes error if the page title isn't equal to the given string STR(page title contains 'title') = causes error if the page title doesn't contain the given string (case-insensitive)

= causes error if the page title doesn't contain the given string (case-insensitive) STR(page url 'url') = causes error if the page url isn't equal to the given string (relative or absolute)

= causes error if the page url isn't equal to the given string (relative or absolute) STR(page url contains 'url') = causes error if the page url doesn't contain the given string

= causes error if the page url doesn't contain the given string STR(next to 'text') = matches the one element that's closest in the DOM to the given text Takes each elem and expands the container around it to its parent, parent's parent etc. until a container containing the text is found - matches that one element associated with that container (matches multiple elems if there's a tie)

= matches the one element that's closest in the DOM to the given text STR(value 'text') = matches if element.value is equal to the given text

= matches if element.value is equal to the given text STR(contains 'text') = matches if the given text is contained in an element's innerText, value, placeholder, associated label's innerText, or currently selected <option>'s innerText (for selects). Case insensitive, leading and trailing whitespace is ignored, and all other whitespace is treated as a single space (in both the 'text' and the DOM).

= matches if the given text is contained in an element's innerText, value, placeholder, associated label's innerText, or currently selected <option>'s innerText (for selects). STR('text') = same as STR(contains 'text')

= same as STR(contains exact 'text') = matches if an element's innerText, value, placeholder, associated label's innerText, or currently selected <option>'s innerText (for selects) equals the given text exactly

= matches if an element's innerText, value, placeholder, associated label's innerText, or currently selected <option>'s innerText (for selects) equals the given text exactly STR(innertext 'text') = matches if an element's innerText contains the given text

= matches if an element's innerText contains the given text STR(selector 'selector') = matches if an element matches the given css selector

= matches if an element matches the given css selector STR(xpath 'xpath') = matches if an element matches the given xpath

= matches if an element matches the given xpath STR(style 'name:value') = matches if an element has the style with the given name set to the given value

= matches if an element has the style with the given name set to the given value STR(position 'N') = matches the one element from the pool of currently matched elements that's in position N (where the first index is 1) Same as ords (1st, 2nd, etc.)

= matches the one element from the pool of currently matched elements that's in position N (where the first index is 1) STR(textbox) = matches if an element is a textbox or textarea

Screenshots When enabled, screenshots are taken before and after every step in branches that include an Open [browser] step. They're a