queues | A queue system for Vapor | Web Framework library

First, if you want the compiler to infer string literal types for the id properties without inferring a readonly tuple type for queues, then you can move the const assertion from the queues initializer to just the id properties in question:

When you invoke clear() on a reference object, it will only clear this particular Reference object without any impact on the rest of your application and no special memory semantics. It’s exactly like you have seen in the code, an assignment of null to a field which has no volatile modifier.

Mind the documentation of clear():

This method is invoked only by Java code; when the garbage collector clears references it does so directly, without invoking this method.

So this is not related to the event of the GC clearing a reference. Your assumption “that all threads will see it cleared” when the GC clears a reference is correct. The documentation of WeakReference states:

Suppose that the garbage collector determines at a certain point in time that an object is weakly reachable. At that time it will atomically clear all weak references to that object and all weak references to any other weakly-reachable objects from which that object is reachable through a chain of strong and soft references.

So at this point, not only all threads will agree that a weak reference has been cleared, they will also agree that all weak references to the same object have been cleared. A similar statement can be found at SoftReference and PhantomReference.

The Java Language Specification, §12.6.2. Interaction with the Memory Model refers to points where such an atomic clear may happen as reachability decision points. It specifies interactions between these points and other program actions, in terms of “comes-before di” and “comes-after di” relationships, the most import ones being:

If r is a read that sees a write w and r comes-before di, then w must come-before di.

If x and y are synchronization actions on the same variable or monitor such that so(x, y) (§17.4.4) and y comes-before di, then x must come-before di.

So, the GC action will be inserted into the synchronization order and even a racy read could not subvert it, but it’s important to keep in mind that the exact location of the reachability decision point is not known to the application. It’s obviously somewhere between the last point where get() returned a non-null reference or refersTo(null) returned false and the first point where get() returned null or refersTo(null) returned true.

For practical applications, the fact that once the reference reports the object to be garbage collected you can be sure that it won’t reappear anywhere¹, is enough. Just keep the reference object private, to be sure that not someone invoked clear() on it.

¹ Letting things like “finalizer resurrection aside”

You can combine QoS - (Quality of service) and Manual ACK to get around this problem. Your exact scenario is documented in https://www.rabbitmq.com/tutorials/tutorial-two-python.html. This example is for python, you can refer other examples as well.

Let says you have 1 publisher and 5 worker scripts. Lets say these read from the same queue. Each worker script takes 1 min to process a message. You can set QoS at channel level. If you set it to 1, then in this case each worker script will be allocated only 1 message. So we are processing 5 messages at a time. No new messages will be delivered until one of the 5 worker scripts does a MANUAL ACK.

If you want to increase the throughput of message processing, you can increase the worker nodes count.

The idea of updating the tables based on message status is not a good option, DB polling is the main reason that system uses queues and it would cause a scaling issue. At one point you have to update the tables and you would bottleneck because of locking and isolations levels.

One can insert a group.next() call inside the loop after reaching a certain count, e.g.:

Even though your configured persistent messages on the payments queue, this just means messages will survive a broker restart - once a message has been consumed and acknowledged it would be removed.

If you know you're going to need the payment_analyze queue at some point in the future, is it viable to just create this queue/binding upfront and route messages to both payment_analyze and payments? Messages on the payment_analyze will bank up until you're ready to start consuming them. Note: If you're producing a large number of messages this approach might result in storage issues...

As an alternative, you could write the messages to BLOB storage (or some other data store) as part of your payments queue consumer (or a different queue/consumer altogether) and then when you're ready to introduce the payment_analyze queue, you could write a script to read all the old messages from BLOB storage and send them to the RabbitMQ exchange. With 'topic' exchanges - see here - you can probably be clever with wildcards and routing keys in your queue bindings to ensure both old messages (from BLOB storage) as well as new messages are both routed to the payment_analyze queue, but only new messages are routed to the payments queue (so that your payments queue consumer is not reprocessing old messages).

Another option (assuming you're not overly invested in RabbitMQ) could be to consider Apache Kafka instead which deals with this scenario quite nicely as messages aren't automatically removed from a partition once they've been processed by a subscriber.

Anyways, just a few options to consider...

I haven't yet implemented an ethernet dext myself, but based on my experience with using DriverKit for other types of drivers and knowing its design goals, I have an idea what the solution might be.

First off, let's clarify: you're implementing either a virtual ethernet device, or you're building a driver for hardware that unites multiple independent ports in one physical device. In the former case (I'm guessing this is what you're doing based on your IOUserClient comment), your driver will be matching IOUserResources and you create the ethernet driver instance when you receive the corresponding message from your user space component.

Now, DriverKit design: DriverKit is built around a goal of keeping the driver instance for each device in its own separate process. Sort of circling back to the microkernel idea. Specifically, this means that multiple devices that use the same driver will each create an independent instance of that driver in its own process. This very likely means that NetworkingDriverKit was never designed to support more than one instance of IOUserNetworkEthernet, because they are seen as separate devices. Hence the crashes.

OK, what do we do about it? Use DriverKit the way it was intended. Put each virtual ethernet adapter in its own driver instance. This gets a bit tricky. I think this should work:

• Your dext has a "control" instance. This is the thing that matches IOUserResources. It's a simple IOService class which listens for the signal (presumably IOUserClient) to create or destroy a virtual ethernet device. When creating a virtual ethernet device, you need that to run in its own driver instance though. (For an actual device with multiple ports, this would match the USB/PCI device nub, manage bus communication with the device, and enumerate the ports.)
• Instead of creating an instance of a IOUserNetworkEthernet subclass, create an instance of a "nub" class and attach it to your central control class instance. Call RegisterService() in its startup code, so it's considered for IOKit matching.
• Set up a second IOKit matching dictionary in your Info.plist, which matches your new "nub" object. Here, use your IOUserNetworkEthernet-derived class to "drive" the nub. Once the virtual ethernet device is ready to use, call RegisterEthernetInterface().

2 extra things to note:

1. The virtual device will run in a separate process from the control object, so they can only communicate via DriverKit inter-process calls, i.e. user clients. Hopefully, they don't really need to communicate much though, and the control client can pass all the information required via properties on the nub. If you're implementing support for multi-port hardware, you probably won't be able to avoid this part though.
2. Your user space component (app, daemon) will need to open a new communication channel with the virtual device, so you'll probably need to implement user client support there too.

I'm not sure how you'd go about shutting down an individual virtual ethernet device, you could try calling Terminate() on either it or the nub it's matched on and see what happens.

You need to deploy artemis-rest.war in etc/bootstrap.xml, e.g.:

As long as the is running, the .then() won't be executed.

This snippet shows the difference in the execution order a bit better:

The field q2.Messages is unreliable, it is the count of messages not awaiting acknowledgment, i.e. messages already ACK'ed.

Your consumer is declared with autoAck = true — i.e. noAck —, which means that no acknowledgements are expected, which means that there are zero messages already ACK'ed.

When you comment out the consumer, the number of acknowledged messages likely depends on the publisher buffer.

Getting a precise number of messages programmatically on a given queue with AMQP 0.9.1 is basically not possible. You may use the message_stats field in the management API instead: