Jordan (Interactively) Scrapes Dota 2 Leagues In Order to Add Admins

Demo Code Here

I love Dota 2. I think it’s one of the best games created. Awesome team work is required and it can be so fulfilling when you work together with your friends and do great.

Honestly, I really love almost all competitive games. Dota scratches the niche perfectly for me so that I can play with friends and family. About five years ago I started a Dota league so I could have a consistent place to play amateur dota matches that meant something. It’s been a really satisfying five years and I’ve met a ton of awesome people through it.

Valve (the creator of Dota) offers a chance for leagues to make their games publicly spectatable from within the game client. Without going in to too many details, it’s a pretty cool thing. The people playing the games would create a custom lobby and, if they had the proper permissions, they could flag the games as these league games.

The big pain we were having was each season we had to submit the captains as admins for this. Sometimes we would have over 100 captains. It was a manually process of copy, paste, submit, click “ok” on the confirmation modal and then repeat. At this point in our league, it was probably our biggest pain point.

So, this is the point of this code. I use Puppeteer to login to Dota2.com/leagues and submit all these admins automatically. It felt so good to just run this and let it do the tedious chore. I also pull the list of admins from a csv file (it has to be the steam url of the admin) so I use the csvtojson in order to easily parse and loop through the list.

At the top of the code I have the variables that are most likely to change easily accessible. The leagueId can be found after you login at dota2.com/leagues and select the league to which you want to add the admins. You can find it in the url (see image below).

That’s your league id up there. Get it!

const leagueId = 10260; // We get 90 seconds to enter our username, password, and 2fa info const timeToLogin = 90000;

We start with an self invoking async / await block to handle the many promises that we will be working with through Puppeteer.

(async () => { // awesome code here })();

Check this for more explanation on async/await.

If you’ve read any of my other posts, you would know this is where I normally have a code block to handle headless and ubuntu set ups. Because valve requires two factor authentication logging in requires user input and therefore cannot be down headless.

We do set up our browser and other variables here, though. We set up a puppeteer browser and set our urls, the first is where we go to login. After we are authenticated, the adminUrl is where we will go to start adding the admins.

const browser: Browser = await puppeteer.launch({ headless: false, args: [`--window-size=${1800},${1200}`] }); const url = 'http://www.dota2.com/league/0/list'; const adminUrl = `http://www.dota2.com/league/${leagueId}/admins`;

Valve requires two factor authentication (I think always?) and so this part requires user input. Starting the script with npm start will (after finishing the install) bring you to the login screen, where it will prompt for your username and password. After you enter those, Valve will send you an email with your two factor authentication code (see image). Enter the code and hit “Submit”. After this, the code will do the rest.

You have to enter your two factor authentication code here.

The above is all executed through the code below. We open a new page and set the viewPort a bit bigger since we’ll need to be interacting with it. It goes to the login screen of dota2.com/leagues and then clicks the signInBox. This takes us to the login screen.

I think it’s noteworthy that we can’t easily navigate directly to the login screen. Amazon does something like this where it has a kind of unique client_id (which I’m guessing is the app that is redirecting to the steam, dota2.com in this case) and I just didn’t want to mess with it. So, we find the button, click it, and then wait.

And wait.

And wait.

Just kidding, this is where you have to do something! Scripts can’t do everything! Maybe one day if I get really ninja I’ll do something to pull the code from my email so it can be completely automated.

const page = await browser.newPage(); await page.setViewport({ height: 1200, width: 1900 }); await page.goto(url); await page.click('.HomeContentSignInBox a'); await page.waitForSelector('.HomeWelcomeText', { timeout: timeToLogin }); await page.goto(adminUrl); await page.waitForSelector('#addAdminForm');

So, it waits for 90 seconds (or if you adjusted the 90000ms above, it would wait that time) and during this time you enter your credentials. Puppeteer has a cool waitForSelector function which pauses the script until it sees the css selector that you designated, .HomeWelcomeText in this case. Once it sees it, it navigates to the admin url (with the proper league id that you set above).

Once here it gets down to business. Now you can go nap. It looks at the csv (here’s a sample) and starts parsing and looping.

const steamUrls = await csv().fromFile('./get_captain_steam_info.csv'); for (let i = 0; i < steamUrls.length; i++) { try { await page.goto(adminUrl); const adminProfileUrl = await page.$('#admin_profile_url'); if (adminProfileUrl) { await adminProfileUrl.click({ clickCount: 3 }); await adminProfileUrl.type(steamUrls[i].steamurl); await page.click('.btn_darkblue_white_innerfade'); await page.waitForSelector('.newmodal_buttons .btn_grey_white_innerfade'); await page.click('.newmodal_buttons .btn_grey_white_innerfade'); await page.waitFor(750); } } catch (e) { console.log('some kind of error doing admin entering', e); } }

What it does is search for the form where we input the admin url and when it finds it, it clicks three times. The form doesn’t clear when it’s submitted and we are going to be doing quite a bit of submitting so we need to clear it ourselves. Puppeteer is so cool in that it can act just like a human user at a browser. It clicks the field three times (once to focus it, twice to highlight one word, three times to highlight all the words) and then types out the steam url that it pulls from the csv file.

Where it puts the admin urls.

C’mon, admit it, that’s pretty cool. I thought it was pretty clever. After it clears the form and enters the url, it submits the form by clicking .btn_darkblue_white_innerfade and then waiting for the “OK” button to appear ( .newmodal_buttons .btn_grey_white_innerfade ) confirming it’s been submitted and clicking it.

Confirmation button.

That’s pretty much it! I love it. Don’t you love it?

Demo Code Here