multidimensional array - Spark scala how to count values in rows -
i new both spark , scala...and have read data file , count value contained in both columns , rows. data set structured this:
0 0 2 0 2 2 0 2 0 2 0 0 0 0 0 0 1 0
in order count number of "2" in each column:
i imported file:
val ip = sc.textfile("/home/../data-scala.txt")
i created array save results
var arraycol = array.ofdim[long](3) val cols = ip.map(line => line.split(" ")) (i <- 0 2) { arraycol(i) = cols.map(col => col(i)).filter(_.contains("2")).count() }
and counted number of "2" contained in each column.
now same each row. have suggestion?
cols.map(r => r.count(_ == "2"))
or shell example:
scala> val cols = sc.parallelize(list("0 1 2", "2 0 2")).map(_.split(" ")) scala> cols.map(_.count(_ == "2")).collect() res1: array[int] = array(1, 2)
Comments
Post a Comment