Cross-Browser and Real-Device Testing with TestingBot Live
On this page
A feature that works in the browser on your desk has been tested on one combination out of the many your users have. Cross-browser testing means repeating the important checks on the other combinations that matter. This chapter covers how to choose them, what to look for on each, and how to run the checks on remote browsers and real devices with TestingBot Live, including on a site that only runs on your laptop.
Choosing the browsers and devices#
You cannot check everything everywhere, so decide on a short matrix and write it down:
- Start from your analytics. List the browser, OS and device combinations that make up most of your traffic, and the ones behind most of your revenue (often not the same: a B2B tool may have most visits from Chrome on Windows and its biggest customers on Safari on Mac).
- Cover each rendering engine. Chrome and Edge use Blink, Firefox uses Gecko and Safari uses WebKit. Every browser on iOS uses WebKit, so "Chrome on iPhone" behaves much more like Safari than like Chrome on Android.
- Add at least one real phone per mobile OS, at a common screen size.
- Add the awkward cases your users actually have: an older OS version still in your support policy, a small Android phone, a tablet in landscape.
A useful split is a primary matrix (two or three combinations where you run the full checklist) and a secondary matrix (everything else in your support policy, where you run a short smoke pass on the critical flows).
What to check on which browser#
Do not re-run functional logic that is browser-independent; a discount is calculated on the server and will not differ between Firefox and Safari. Spend the time on what does differ:
- Layout and CSS. Newer CSS features, flex and grid gaps, sticky headers, fonts and line breaks, overflowing text in long translations.
-
Native form controls. Date and time pickers,
selectmenus, file inputs, number fields, autofill. These are drawn by the browser and OS and behave differently on each. - JavaScript APIs. Clipboard, notifications, payment request, media playback and autoplay rules, storage limits in private browsing.
- Input. Keyboard navigation and focus outlines, hover-only menus on touch screens, the on-screen keyboard covering inputs on phones.
- Cookies and third-party content. Safari blocks third-party cookies by default, which breaks some embedded logins, payment frames and chat widgets.
-
Media and scrolling. Video playback, smooth-scroll, scroll snapping,
position: fixedelements while the mobile address bar shows and hides.
You have time for one extra browser check before release. Your matrix already covers Chrome on Windows and Chrome on Android. Which addition adds the most coverage?
Edge and Chrome on macOS both use Blink, the same engine you already test. Safari on iOS is WebKit, on a different OS, with touch input and its own cookie and media rules, so it is the most likely of the three to behave differently.
Running the checks in TestingBot Live#
TestingBot Live gives you an interactive remote browser or mobile device, controlled with your own mouse and keyboard from inside your browser. There is nothing to install. The browsers available are the current and recent versions of Chrome, Firefox, Safari and Microsoft Edge on Windows, macOS and Linux, plus mobile Safari on real iPhones and iPads and mobile Chrome on real Android devices and emulators; the browser list shows the exact versions. Each session runs on a freshly provisioned virtual machine that is destroyed when the session ends, so you start every session from a clean browser profile.
A typical session:
- In the member area, open Live testing and pick the browser, version and operating system (or a device).
- Enter the URL to test and start the session. The remote desktop or device screen appears in your browser tab.
- Work through your checklist or charter. The browser's own developer tools are available with F12 or the usual shortcut, so you can read the console and network panel while you test.
- Use the session toolbar for the things you cannot do with a mouse alone:
- Resolution changes the screen resolution of a desktop session, useful for layouts that break at a particular width.
- Copy/Paste syncs your local and remote clipboard, so you can paste long test data such as addresses in other scripts.
- Change GPS sets a mock GPS position, and, on paid plans, Change Location restarts the session so the browser appears to come from another country, for geo-dependent prices and content.
- Device Settings switches between light and dark mode.
- Rotate turns a mobile device between portrait and landscape.
- Report Bug takes an annotated screenshot and can file it straight into a connected issue tracker (see the previous chapter).
- Share creates a view-only link so a developer or designer can watch the session live while you reproduce the bug.
- Stop the session when you are done, or switch to the next combination in your matrix.
Two shortcuts save time once your matrix is fixed:
-
Start from a link. A URL of the form
https://testingbot.com/members/manual/start?url=<url>&browserName=safari&version=latest&platform=TAHOEopens a confirmation page with that page and browser preselected; click Start session to begin. Put such links in a bug report so the developer opens the exact browser it was reported on, or in a pull request so reviewers check the preview deploy. Details and a CI example are on the Start from a link page. - The browser extension. The TestingBot extension for Chrome and Firefox opens the page you are looking at on a remote browser of your choice, in two clicks.
Note the plan limits: a free trial includes 60 minutes of Live testing in total, with 5 minutes per session. That is enough to try the workflow on a couple of combinations; the Live testing FAQ lists the details.
A customer reports that the price on your product page is shown in the wrong currency when they visit from Germany. How do you reproduce it in a Live session?
If the site picks the currency from the visitor's location, you need the request to come from Germany, which is what Change Location does (it restarts the session, and needs a paid plan). Changing the browser language tests the language preference, a different input that may or may not drive currency.
Testing localhost and staging through the tunnel#
A remote browser cannot reach http://localhost:3000 on your laptop or a staging server behind your firewall on its own. TestingBot Tunnel fixes that: it runs on a machine inside your network and carries the remote browser's requests over an encrypted connection.
-
Start your site and check it loads locally, for example on port 3000.
-
Start the tunnel on the same machine and wait for
You may start your tests:java -jar testingbot-tunnel.jar -
In the Live launcher, keep the tunnel selected under Session options (your running tunnels are listed there, and the first one is preselected).
-
Open
http://local.testingbot.com:3000rather thanhttp://localhost:3000.local.testingbot.comresolves to 127.0.0.1, so through the tunnel it means "the machine running the tunnel", and unlikelocalhostit works on every port and on mobile devices.
If several testers need the same staging environment, one shared tunnel on a server inside the network (started with --shared) appears in everyone's tunnel list, so non-developers never have to run Java themselves. The Local Testing with a Secure Tunnel course covers the tunnel in depth, including the manual testing chapter and troubleshooting.