HScan | 超精简的POC扫描框架 | Security Testing library

I'm using Redis with hashes to index real paths from file system, e.g.:

While working with Redis using Spring Data Redis, I tried to scan hash data in my server (similar to HSCAN in CLI) -

I was trying to scan for particular key stored in JedisCluster.

I am using spring boot (irrelevant) with spring-data-redis:jar:2.0.9, which uses lettuce to connect to my REDIS. I am using a hash structure that contains around 100 keys. Under those keys I put some objects which type is also irrelevant:

I currently have a scenario where we are using REDIS to store string field-value pairs within a hashed set HSET.

The original reasoning behind using hashed sets instead of just sets is ease of retrieving the records using HSCAN inside a GUI Search Bar as opposed to just SCAN because it's easier to get the length of a hash to use in the COUNT field.

I read in the Redis documentation that both GET and HGET commands execute with O(1) time complexity, but a member of my team thinks that if I store all the values inside a single key then it basically returns the entire hash during HGET instead of the singular field-value that I need.

So for a made up but similar example:

• I have a Redis instance with a single Hashed Set called users.
• The hashed set has 150,000 field:value pairs of username:email

If when I execute hget users coolguy, is the entire hash getting returned or just the email for user coolguy?

I have database items that, in addition to their primary key, need an index unique to the group in which the items belong. Let's call the property nbr, and the property that groups items together and defines the scope of unique nbr:s we'll call group. This nbr must be in the [1-N] range, and may be set when items are imported from an external source. Since all items must have a nbr, the task then becomes how to track which values are used, to enable picking a free nbr for new items that are added manually.

I'm using DynamoDB and Redis. I cannot have a DynamoDB index on nbr. The idea I have so far is to use Redis to track which numbers have been used for a particular group, so that for a Redis key such as -item-nbrs I could store all the used nbr:s and implement logic to find the next free nbr. Holes in the range of used nbr is acceptable, but holes should be filled before considering the numbers exhausted.

Essentially I want to find unused indices of a sparse array of max size N.

What would be a good structure for storing this information in Redis to quickly find a free nbr? My ideas so far include:

• A single comma-separated string of all used nbr's in sorted order? To find a free nbr, a GET command is issued and the string is parsed until a hole is found or the end of the list, the picked number is inserted into the string and then the entire string is replaced. When N is large, this seems very inefficient.

• A hash where each used nbr is stored as its own field, and using e.g. HSCAN to iterate through the fields of the hash to find a free nbr. When N is large, the HSCAN must scan a lot of fields.

• Partitioning my nbr:s into fields called say p1-20, p21-40, p41-60, ..., each containing a sorted set of the used nbr:s within that partition only, and when a partition is exhausted (no more free nbr:s), remove it completely to speed up further iteration. Use HSCAN to iterate, and HSET to start a new partition.

• Storing all free nbr instead of all used, and using sorted sets and ZPOPMIN or regular lists and LPOP, possibly partitioned into sub-sets. Pre-populating Redis with all free nbr 1-N seems ugly though.

Let's say N is in the magnitude of 65536.

Are any of the solutions above better/worse, for performance or other reasons? Is there a better/smarter way, maybe utilizing some clever aspect of Redis that I'm unaware of?

Edit:

Kevin's answer resulted in the following solution (pseudo code):

I'm starting to work with a Redis database and I have been querying a key that is of type hash. I do an HSCAN to check some values and its format. I get the following results.

I was wondering if there is any way I can know what's the representation of this data. Is it hexadecimal, binary?. I was also wondering if there is any way to decode this, either by a client or some sort of Python script (I have seen some around but I don't know if they apply to this case)

In Redis, I have hash keys in the following format

I am trying to do an HSCAN using Lettuce's synchronous commands. The problem is that I can't figure out the correct way of initializing the MapScanCursor. I've had no success with the constructor, and MapScanCursor.INITIAL gives type ScanCursor (no luck casting that to MapScanCursor either).

Here is an example: