javascript - Scrape html from js with node.js and horseman -
i trying scrape array of batters salary information page: https://www.swishanalytics.com/optimus/mlb/dfs-batter-projections
i using node.js , node-horseman.
here code:
var horseman = require('node-horseman'); var horseman = new horseman(); horseman.open('https://www.swishanalytics.com/optimus/mlb/dfs-batter-projections'); if (horseman.status() === 200) { console.log('[+] successful page opening') horseman.screenshot('image.png'); console.log(horseman.html()); } horseman.close();
the issue return horseman.html() still lot of javascript , cannot extracted cheerio. how can execute javascript programatically?
for example, if view source @ same link see area has batters starts
function model(){ this.batterarray = [{"team_short":"rockies","mlbam_id":"571448","player_name":"nolan arenado",
obviously still javascript... i'm assuming @ point must executed , converted html presented browser?
i tested out , seems work:
var horseman = require('node-horseman'); var horseman = new horseman(); horseman.open('https://www.swishanalytics.com/optimus/mlb/dfs-batter-projections'); if (horseman.status() === 200) { console.log('[+] successful page opening') horseman.screenshot('image.png'); var batters = horseman.evaluate(function(){ return (new model()).batterarray; }); console.log(batters); } horseman.close();
that give array of batters can use in code. write out file or create table out of it.
Comments
Post a Comment