msgpack-ruby | MessagePack implementation for Ruby / msgpack.org | Performance Testing library
kandi X-RAY | msgpack-ruby Summary
kandi X-RAY | msgpack-ruby Summary
MessagePack implementation for Ruby / msgpack.org[Ruby]
Support
Quality
Security
License
Reuse
Top functions reviewed by kandi - BETA
Currently covering the most popular Java, JavaScript and Python libraries. See a Sample of msgpack-ruby
msgpack-ruby Key Features
msgpack-ruby Examples and Code Snippets
Community Discussions
Trending Discussions on msgpack-ruby
QUESTION
I am working on a data system that needs to store large amounts of simple, extensible data (alongside some specialist indexing we are developing in-house, and not part of this question). I expect there to be billions of records stored, so efficient serialisation is a key part of the system. The serialisation needs to be fast, space-efficient, and supported in multiple platforms and languages (because packing and unpacking this data will be a client component responsibility, not part of the storage system)
The data type is effectively a hash with optional key/value pairs. Keys will be small integers (interpreted at application layer). Values can be a variety of simple data types - String, Integer, Float.
As a technology choice, we have picked MessagePack, and I am writing code to perform data serialisation via Ruby's msgpack-ruby gem.
I don't need the precision of Ruby's 64-bit Float. None of the numbers being stored has meaningful precision even to limits of 32-bit. So I want to use MessagePack support for 32-bit floating point values. This definitely exists. However, the default behaviour in Ruby on any 64-bit system is to serialise Float to 64 bits:
...ANSWER
Answered 2018-Sep-07 at 12:29As of right now (version 1.2.4 of msgpack-ruby
) this is not possible in the exact fashion you tried: the msgpack_packer_write_value
function first checks all hard-coded data types, and handles them with its default implementation. Only if the current object does not fit any of those types are the extensions handled.
In other words: you cannot override the default pack formats with MessagePack::DefaultFactory#register_type
, calling that will simply be a no-op.
Furthermore, the extension mechanism is not what you are looking at, anyways. Using that, messagepack would emit a marker byte "this is an extension", followed by the extension ID (the value "0" in your example), followed by what is already encoded as float32 - alternatively you would need to handle the binary encoding/decoding yourself.
Creating your own Float classYou could, in principle, create your own FloatX
class or whatever, but this is just a very bad move:
Float
has nonew
method which you could monkeypatch, and I know of no way to tell ruby to create aFloatX
instance when you write10.3
in your code. So you would have to do manual object creation throughout your code, probably with severe impact on performance.- You would end up with the extension mechanism anyways, infeasible as shown above.
msgpack_packer_write_value
You would need to to override the msgpack_packer_write_value
implementation of packer.c
. Unfortunately you cannot do that in the ruby world since there is no equivalent ruby method defined for it. So the usual monkeypatching of ruby cannot be used.
Also, the method is called from plenty of other methods inside the packer.c
implementation, for example in the respective methods responsible for writing arrays or hashes. Those of course would not call a ruby method of the same name either, as they're living in their binary world completely.
Finally, whily the usage of a factory mechanism seems to imply that you can somehow create different implementations of packers, I see no evidence that this is actually true - reading the C code of the Gem, there seems to be no provision for anything of that kind. The factory seems to be there to handle the ruby<->C interactions of the Gem.
What nowIf I were in your shoes, I would clone that Gem and modify msgpack_packer_write_value
in packer.c
to behave as you wish. Check the case T_FLOAT
and go on from there. The code seems pretty straightforward - it soon proceeds to the following method in packer.h
:
Community Discussions, Code Snippets contain sources that include Stack Exchange Network
Vulnerabilities
No vulnerabilities reported
Install msgpack-ruby
To build -java gems for JRuby, run:. If this directory has Gemfile.lock (generated with MRI), remove it beforehand.
Support
Reuse Trending Solutions
Find, review, and download reusable Libraries, Code Snippets, Cloud APIs from over 650 million Knowledge Items
Find more librariesStay Updated
Subscribe to our newsletter for trending solutions and developer bootcamps
Share this Page