Decisions | A simple decision making app for android

Assuming the @ and # only appear at the end of string in front of the author or category, you can use

Great question. As you say, there are no hard and fast rules here and opinions do vary, but let me share my perspective as a QB data scientist and kedro maintainer who has used the layering convention you referred to several times.

For a start, let me emphasise that there's absolutely no reason to stick to the data engineering convention suggested by kedro if it's not suitable for your needs. 99% of users don't change the folder structure in data. This is not because the kedro default is the right structure for them but because they just don't think of changing it. You should absolutely add/remove/rename layers to suit yourself. The most important thing is to choose a set of layers (or even a non-layered structure) that works for your project rather than trying to shoehorn your datasets to fit the kedro default suggestion.

Now, assuming you are following kedro's suggested structure - onto your questions:

When is a dataset a feature rather than a primary dataset? The distinction seems vague...

In the case of simple features, a feature dataset can be very similar to a primary one. The distinction is maybe clearest if you think about more complex features, e.g. formed by aggregating over time windows. A primary dataset would have a column that gives a cleaned version of the original data, but without doing any complex calculations on it, just simple transformations. Say the raw data is the colour of all cars driving past your house over a week. By the time the data is in primary, it will be clean (e.g. correcting "rde" to "red", maybe mapping "crimson" and "red" to the same colour). Between primary and the feature layer, we will have done some less trivial calculations on it, e.g. to find one-hot encoded most common car colour each day.

Is it OK for a primary dataset to consume data from another primary dataset?

In my opinion, yes. This might be necessary if you want to join multiple primary tables together. In general if you are building complex pipelines it will become very difficult if you don't allow this. e.g. in the feature layer I might want to form a dataset containing composite_feature = feature_1 * feature_2 from the two inputs feature_1 and feature_2. There's no way of doing this without having multiple sub-layers within the feature layer.

However, something that is generally worth avoiding is a node that consumes data from many different layers. e.g. a node that takes in one dataset from the feature layer and one from the intermediate layer. This seems a bit strange (why has the latter dataset not passed through the feature layer?).

Is it good practice to build a feature dataset from the INT layer? or should it always pass through Primary?

Building features from the intermediate layer isn't unheard of, but it seems a bit weird. The primary layer is typically an important one which forms the basis for all feature engineering. If your data is in a shape that you can build features then that means it's probably primary layer already. In this case, maybe you don't need an intermediate layer.

The above points might be summarised by the following rules (which should no doubt be broken when required):

1. The input datasets for a node in layer L should all be in the same layer, which can be either L or L-1
2. The output datasets for a node in layer L should all be in the same layer L, which can be either L or L+1

If anyone can offer any further advice or blogs\docs talking about Kedro Data Modelling that would be awesome!

I'm also interested in seeing what others think here! One possibly useful thing to note is that kedro was inspired by cookiecutter data science, and the kedro layer structure is an extended version of what's suggested there. Maybe other projects have taken this directory structure and adapted it in different ways.

Yes, by default GKE sends logs for all pods to Stackdriver and you can view/query them there.

Make it more detailed. You can say something like This identifier will be used to collect Crash Data and in-app activity in order to improve functionalities and user engagement. Or something alike.

In your String you only make reference to Crashlytics but you're missing reference toAnalytics.

It is possible that Apple answers back saying something like they didn't find the Alert in your app after doing the correction of the String. If that happens, you just have to answer them that the alert only shows once per device (if so), and the class where you display the alert (commonly used in the AppDelegate).

Currently your function limited to a certain depth. If you really want to write your own function, you would have to use a recursive function.

Anyway - there a better solutions. If you decoded the json by yourself, you can tell the decode function to return an array:

The data in each of those topics will be retained as per a property log.retention.{hours|minutes|ms}. At the Kafka server level, this is set to 7 days by default for all topics. You could change this at a topic level as well.

In such a setting, a consumer will not be able to read the entire history if it needed to, so in this instance typically a consumer would:

1. consume the message i.e. "a marble no. 5 was added to red jar" at offset number 5
2. carry out the increment step i.e. redCount++ and store the latest information (redCount = 5) in a local state store
3. Then commit the offset back to Kafka telling that it has read the message at offset number 5
4. Then, just wait for the next message...

If however, your consumer doesn't have a local state store - In this case, you would need to increase the retention period i.e. log.retention.ms=-1 to store the data forever. You could configure the configure the consumers to store that information locally in memory but in the event of failures there would be no choice but for the consumers to read from the beginning. This I agree is inefficient.

The selection parameter of Picker will do the work of storing the tip amount for you -- there's no need for the tipSelected = line of imperative code.

Then, unless you're planning on mutating them somewhere, the if_10 and if_20 don't really need to be @State variables.

Here's one possible solution:

The pattern I follow is rather to have a bloc that handles the logic, in your example that would be the Auth check. In the bloc I'd emit a new state, say a state called Authed, when if (isAuthed) pass.

In the view you could have a BlocListener that trigger on the state change to Authed and do your routing there or display a Snackbar or similar.

This way you don't have to mix view actions with logic that really doesn't require view information.

Your first stops for further reading should probably be:

• What Every Programmer Should Know About Memory? re: cache
• https://agner.org/optimize/ re: everything else about writing efficient asm.
• https://uops.info/ for a better version of Agner Fog's instruction tables.
• See also other links in https://stackoverflow.com/tags/x86/info

so I will put out microoptimalisation until it will suffer performance problems and then I will start with profiling.

Yes, probably good to start trying stuff so you have something to profile with HW performance counters, so you can correlate what you're reading about performance stuff with what actually happens. And so you get some ideas of possible details you hadn't thought of yet before you go too far into optimizing your overall design idea. You can learn a lot about asm micro-optimization by starting with something very small scale, like a single loop somewhere without any complicated branching.

Since modern CPUs use split L1i and L1d caches and first-level TLBs, it's not a good idea to place code and data next to each other. (Especially not read-write data; self-modifying code is handled by flushing the whole pipeline on any store too near any code that's in-flight anywhere in the pipeline.)

Related: Why do Compilers put data inside .text(code) section of the PE and ELF files and how does the CPU distinguish between data and code? - they don't, only obfuscated x86 programs do that. (ARM code does sometimes mix code/data because PC-relative loads have limited range on ARM.)

Yes, making sure all your data allocations are nearby should be good for TLB locality. Hardware normally uses a pseudo-LRU allocation/eviction algorithm which generally does a good job at keeping hot data in cache, and it's generally not worth trying to manually clflushopt anything to help it. Software prefetch is also rarely useful, especially in linear traversal of arrays. It can sometimes be worth it if you know where you'll want to access quite a few instructions later, but the CPU couldn't predict that easily.

AMD's L3 cache may use adaptive replacement like Intel does, to try to keep more lines that get reused, not letting them get evicted as easily by lines that tend not to get reused. But Zen2's 512kiB L2 is relatively big by Forth standards; you probably won't have a significant amount of L2 cache misses. (And out-of-order exec can do a lot to hide L1 miss / L2 hit. And even hide some of the latency of an L3 hit.) Contemporary Intel CPUs typically use 256k L2 caches; if you're cache-blocking for generic modern x86, 128kiB is a good choice of block size to assume you can write and then loop over again while getting L2 hits.

The L1i and L1d caches (32k each), and even uop cache (up to 4096 uops, about 1 or 2 per instruction), on a modern x86 like Zen2 (https://en.wikichip.org/wiki/amd/microarchitectures/zen_2#Architecture) or Skylake, are pretty large compared to a Forth implementation; probably everything will hit in L1 cache most of the time, and certainly L2. Yes, code locality is generally good, but with more L2 cache than the whole memory of a typical 6502, you really don't have much to worry about :P

Of more concern for an interpreter is branch prediction, but fortunately Zen2 (and Intel since Haswell) have TAGE predictors that do well at learning patterns of indirect branches even with one "grand central dispatch" branch: Branch Prediction and the Performance of Interpreters - Don’t Trust Folklore