Simplest way to get the top n elements of a Scala Iterable

Is there a simple and efficient solution to determine the top n elements of a Scala Iterable? I mean something like iter.toList.sortBy(_.myAttr).take(2) but without having to sort all elements...

document.write not working in html

I am try to write inside html document. this is my javascript code: <script type="text/javascript">document.write("Hello World!")</script> I am working with chrome and get the following...

Scala Stream call-by-need (lazy) vs call-by-name

So I understand that call-by-need is simply a memoized version of call-by-name. In Martin Odersky's FP Course on Coursera, in lecture 7.3 (Lazy Evaluation), he mentions that if Streams were...

Difference between >> and >>> in Scala

Is there any difference between >> and >>> operator in Scala? scala> 0x7f >>> 1 res10: Int = 63 scala> 0x7f >> 1 res11: Int = 63 scala> 0x7f >> 4 res12: Int = 7 scala> 0x7f >>> 4 res13: Int = 7

Typesafe config: How to iterate over configuration items

In my Play application I've a configuration like this: social { twitter { url="https://twitter.com" logo="images/twitter.png" } facebook { ...

sbt fails with java.lang.NoClassDefFoundError: sbt/ConsoleOut$

After using sbt for some time, something failed and I can't run sbt on my Ubuntu machine. I get this exception no matter what I do. Even running sbt help causes the following...

Render two json objects to the view with Play! scala 2.3

Here is a dumb question : I simply want to render two Json object to the view, I manage to do render them separately but not together... I tried a simple trick like that (I tried with two '+' as...

Building Scala Object, Class Not Found

I am trying to build my Scala Project with Maven, I have used this sample POM as a guide line: http://ryancompton.net/2014/05/19/sample-pomxml-to-build-scala--jar-with-dependenciesjar/ But I am...

Slick - Update full object or more than 22 columns

I've a table user_permissions which has 46 permission columns along with id and created_date. This table has a corresponding UserPermissions class: class UserPermission(val id: Long, val...

spray json implicit UUID conversion

I have a User model case class User(name: String, email: String, password: Option[String] = None, key: Option[UUID] = None) With a spray-json marshaller object UserJsonSupport extends...

load a local file to spark using sc.textFile()

#Question# How to load a file from the local file system to Spark using sc.textFile? Do I need to change any -env variables? Also when I tried the same on my windows where Hadoop is not installed...

Parsing large JSON file with Scala and JSON4S

I'm working with Scala in IntelliJ IDEA 15 and trying to parse a large twitter record json file and count the total number of hashtags. I am very new to Scala and the idea of functional...

Compile error on a Scala project: object _ is not a member of package _

I got seemingly simple problem.. and usually I could find a solution easily from stackoverflow, but I had no luck this time. Here's problem. I have created a object in a package called...

How to list all cassandra tables

There are many tables in cassandra database, which contain column titled user_id. The values user_id are referred to user stored in table users. As some users are deleted, I would like to delete...

Read from a hive table and write back to it using spark sql

I am reading a Hive table using Spark SQL and assigning it to a scala val val x = sqlContext.sql("select * from some_table") Then I am doing some processing with the dataframe x and finally...

Coursera Scala SBT submission issues

I am a current student at the EPFL functional programming in SCALA course on coursera. I have successfully passed the first course by submitting my assignments via sbt with no issues until...

sparkR: Error while instantiating 'org.apache.spark.sql.hive.HiveSessionState':

I am struggling solving this problem when I try to use sparkR. sparkR.session(master = "local[*]", sparkConfig = list(spark.driver.memory = "1g")) Error in handleErrors(returnStatus, conn) : ...

Spark Sql: Loading the file from excel sheet (with extension .xlsx) can not infer the schema of a date-type column properly

I have a xlsx file containing date/time filed (My Time) in following format and sample records - 5/16/2017 12:19:00 AM 5/16/2017 12:56:00 AM 5/16/2017 1:17:00 PM 5/16/2017 5:26:00 PM 5/16/2017...

json4s is faling with NPE, what can be the proper format for nested json

var json= {"a1":"b","c1":"d","e1":{"f1":"g","h1":"i","j1":"k"}} implicit val formats = org.json4s.DefaultFormats parse(json).extract[Map[String, String]] I am getting NullPointerException when i...

Schema comparison of two dataframes in scala

I am trying to write some test cases to validate the data between source (.csv) file and target (hive table). One of the validation is the Structure validation of the table. I have load the .csv...

How to update few records in Spark

i have the following program in Scala for the spark: val dfA = sqlContext.sql("select * from employees where id in ('Emp1', 'Emp2')" ) val dfB = sqlContext.sql("select * from employees where id...

Scala Int overflow

Can anyone please help me to understand that why Int type overflow result is not consistent with higher numbers. Sometime it is positive, sometime negative and finally converge to zero. import...

Scala classes don't get load after upgrading to Scala 2.12 version in Intellij

I've upgraded my Scala version from 2.11+ to 2.12. But since then, all my Gatling tests crash in Intellij. I'm using version 2017.3. I get this...

SparkML - Getting several metrics at the same time with RegressionEvaluator()

I am trying to evaluate a Gradient-Boosted Tree Regression model using RegressionEvaluator(). I would like to compare the four possible metrics for this evaluator: rmse mse r2 mae This is how I...

Need to convert a Seq[Option[A]] to Option[Seq[A]]

USE CASE I have a list of files that can might have a valid mime type or not. In my code, I represent this using an Option. I need to convert a Seq[Option[T]] to Option[Seq[T]] so that I do not...

How can I mock DynamoDB access via Spark in Scala?

I have a Spark job written in Scala that ultimately writes out to AWS DynamoDB. I want to write some unit tests around it, but the only problem is I don't have a clue how to go about mocking the...

How to remove null values in list of objects using Circe

I'm trying to encode a list of objects using Circe, something that looks similar to: val test = Seq(MyObject("hello", None, 1, 2, None) I'm trying to parse this using Circe: test.asJson But this...

Why are not class parameters not being called on the Object in Scala?

In Scala how are not class parameters not truly the values being called on the object? The class parameters are part of primary constructor, so should it not be setting up the member variables of...

What's the equivalent of `scala.tools.nsc.MainGenericRunner` in Scala 3?

what's the equivalent of scala.tools.nsc.MainGenericRunner in dotty? I checked in the package dotty.tools.dotc https://github.com/lampepfl/dotty/tree/master/compiler/src/dotty/tools/dotc but...

How to Read data from kafka topic with different schema (has some optional objects) in structured streaming

i have data coming in kafka topic which has an optional object , and since its optional i am missing those records when reading with a defined schema ex : schema i have : val schema =new...