upickle | uPickle: a simple, fast, dependency-free JSON & Binary (MessagePack) serialization library for Scala | Serialization library

Regarding your first question, I assume, your are interested in good IDE support, e.g. completion and jump-to the sources of your dependencies.

Mill already supports IDE integration. It comes with a project generator for IntelliJ IDEA (mill mill.scalalib.GenIdea/idea), which automatically downloads the sources for you. Alternatively, you can use the new BSP Support (Build Server Protocol) which should in combination with the Metals Language Server (https://scalameta.org/metals/) provide a nice editing experience in various IDEs and Editors. Unfortunately, at the time of this writing, Mills built-in BSP server isn't as robust as its IDEA generator, but there is even another alternative, the Bloop contrib module. All these methods should provide decent code navigation through dependencies and completion.

And to your second question:

Is it possible to define that a module is available only in test (like org.scalatestplus.play in the previous code) or should I create separate ivyDeps for testing module?

Test dependencies are declared it the test modules (which are technically regular modules too).

The outer element of your JSON is not an array, it is an object with a single element dataflows whose value is an array. Try jsonData("dataflows")(0).

Idiomatic or scala way to do this by using scala's Option.

Fortunately, upickle Values offers them. Refer strOpt method in this source code.

Your problem in code is str methods in m("always").str and m("sometimes").str With this code, you are prematurely assuming that all the values are strings. That's where the strOpt method comes. It either outputs a string if its value is a string or a None type if it not. And we can use getOrElse method coupled with it to decide what to throw if the value is None.

Following would be the optimum way to handle this.

Take a look at following files, you should get some ideas!

CborAkkaSerializer.scala

LocationAkkaSerializer.scala

Note: These serializers are using cbor

It seems you have both Spark 2.x and 3.x jars in classpath. According to the sbt file, Spark 2.x should be used, however, JsonFileFormat was added in Spark 3.x with this issue

You are creating Value anew every time you call r so, every changes you would make to it, are dismissed.

You create one copy when you call println(r).

Then you create a separate copy with insert(r, "b", transform(None).to(Value)), mutate it and dismiss.

Then you are creating third copy with another println(r).

If you want to refer to the same object use val instead of def.

You can't override defs in a tasks. Just locally defining some ivyDeps and compileIvyDeps will not magically make super.assembly using them.

Of course you can create that task by looking how super.assembly is defined in JavaModule, but you will end up copying and adapting a lot more targets (upstreamAssembly, upstreamAssemblyClasspath, transitiveLocalClasspath, and so on) and make your buildfile hard to read.

A better way would be to make the lighter dependencies and assembly rules the default and move the creation of the standalone JAR into a sub module.