multidimensional array - Spark scala how to count values in rows -


i new both spark , scala...and have read data file , count value contained in both columns , rows. data set structured this:

0 0 2 0 2 2 0 2 0 2 0 0 0 0 0 0 1 0 

in order count number of "2" in each column:

i imported file:

val ip = sc.textfile("/home/../data-scala.txt") 

i created array save results

var arraycol = array.ofdim[long](3)  val cols = ip.map(line => line.split(" "))  (i <- 0 2) {     arraycol(i) = cols.map(col => col(i)).filter(_.contains("2")).count() } 

and counted number of "2" contained in each column.

now same each row. have suggestion?

cols.map(r => r.count(_ == "2")) 

or shell example:

scala> val cols = sc.parallelize(list("0 1 2", "2 0 2")).map(_.split(" ")) scala> cols.map(_.count(_ == "2")).collect()  res1: array[int] = array(1, 2) 

Comments

Popular posts from this blog

Fail to load namespace Spring Security http://www.springframework.org/security/tags -

sql - MySQL query optimization using coalesce -

unity3d - Unity local avoidance in user created world -