javascript - Scrape html from js with node.js and horseman -


i trying scrape array of batters salary information page: https://www.swishanalytics.com/optimus/mlb/dfs-batter-projections

i using node.js , node-horseman.

here code:

var horseman = require('node-horseman'); var horseman = new horseman();  horseman.open('https://www.swishanalytics.com/optimus/mlb/dfs-batter-projections');  if (horseman.status() === 200) {     console.log('[+] successful page opening')     horseman.screenshot('image.png');     console.log(horseman.html());   }     horseman.close(); 

the issue return horseman.html() still lot of javascript , cannot extracted cheerio. how can execute javascript programatically?

for example, if view source @ same link see area has batters starts

function model(){ this.batterarray =  [{"team_short":"rockies","mlbam_id":"571448","player_name":"nolan arenado", 

obviously still javascript... i'm assuming @ point must executed , converted html presented browser?

i tested out , seems work:

var horseman = require('node-horseman'); var horseman = new horseman();  horseman.open('https://www.swishanalytics.com/optimus/mlb/dfs-batter-projections');  if (horseman.status() === 200) {     console.log('[+] successful page opening')     horseman.screenshot('image.png');     var batters = horseman.evaluate(function(){         return (new model()).batterarray;     });     console.log(batters);   }     horseman.close(); 

that give array of batters can use in code. write out file or create table out of it.


Comments

Popular posts from this blog

Fail to load namespace Spring Security http://www.springframework.org/security/tags -

sql - MySQL query optimization using coalesce -

unity3d - Unity local avoidance in user created world -