CatLib | CatLib for unity3d dependency injection framework | Dependency Injection library

You can share fixtures between files using References as is shown in the docs: https://symfony.com/doc/current/bundles/DoctrineFixturesBundle/index.html#sharing-objects-between-fixtures

In your first file do something like this:

When you manually split a string line from CSV, you get an array of strings. Then you have to manually parse the values from strings. Java doesn't handle it automatically.

In your case to handle floats you need to change clr.setDueToValue(strArr[5]) to clr.setDueToValue(Float.parseFloat(strArr[5])), see the doc.

Similarly you can use Integer.parseInt() to parse integers.

For parsing dates you will likely need to use a SimpleDateFormat.

The PCollections you have (record8 and record3) have incompatible triggers. One is Repeatedly.forever, while another is DefaultTrigger. Which means that Beam cannot reason about what records will be available at which trigger firings, so it cannot produce meaningful JOIN results.

One way to fix this is to set both PCollections to Repeatedly.forever(...) trigger, see Setting the triggers section in the Beam guide.

When you have both inputs set to the same trigger, Beam SQL will perform joins and groupbys once per trigger firing.

Version of Beam you are using is likely force-setting the Repeatedly.forever(...) when you use the GROUP BY clause. This behavior is changed in the upcoming Beam version, GROUP BY clause will no longer override the trigger, but it will require both unbounded inputs to have the DefaultTrigger.

I'd say go for

1> what shall I implement in ExtractObjectsFn() so that the records gets converted into BeamRecords ?

In the processElement() method of ExtractObjectsFn, you simply need to convert a CSV line from the input (String) to a Clorox type. Split the string by the comma delimiter (,), which returns an array. Iterate over the array to retrieve the CSV values and construct the Clorox object.

2> How to write the final output to a csv file ?

Similar process as above. You simply need to apply a new transform that will convert a BeamRecord to a CSV line (String). Members of the BeamRecord can be concatenated into a string (CSV line). After this transform is applied, the TextIO.Write transform can be applied to write the CSV line to file.