kandi background
Explore Kits

preferable | User preference management for ActiveRecord | Widget library

 by   bsm Ruby Version: Current License: No License

 by   bsm Ruby Version: Current License: No License

Download this library from

kandi X-RAY | preferable Summary

preferable is a Ruby library typically used in User Interface, Widget applications. preferable has no bugs, it has no vulnerabilities and it has low support. You can download it from GitHub.
Simple filtering for ActiveRecord. Sanitizes simple and readable query parameters -great for building APIs & HTML filters.
Support
Support
Quality
Quality
Security
Security
License
License
Reuse
Reuse

kandi-support Support

  • preferable has a low active ecosystem.
  • It has 9 star(s) with 1 fork(s). There are 2 watchers for this library.
  • It had no major release in the last 12 months.
  • preferable has no issues reported. There are no pull requests.
  • It has a neutral sentiment in the developer community.
  • The latest version of preferable is current.
preferable Support
Best in #Widget
Average in #Widget
preferable Support
Best in #Widget
Average in #Widget

quality kandi Quality

  • preferable has 0 bugs and 0 code smells.
preferable Quality
Best in #Widget
Average in #Widget
preferable Quality
Best in #Widget
Average in #Widget

securitySecurity

  • preferable has no vulnerabilities reported, and its dependent libraries have no vulnerabilities reported.
  • preferable code analysis shows 0 unresolved vulnerabilities.
  • There are 0 security hotspots that need review.
preferable Security
Best in #Widget
Average in #Widget
preferable Security
Best in #Widget
Average in #Widget

license License

  • preferable does not have a standard license declared.
  • Check the repository for any license declaration and review the terms closely.
  • Without a license, all rights are reserved, and you cannot use the library in your applications.
preferable License
Best in #Widget
Average in #Widget
preferable License
Best in #Widget
Average in #Widget

buildReuse

  • preferable releases are not available. You will need to build from source code and install.
  • Installation instructions, examples and code snippets are available.
preferable Reuse
Best in #Widget
Average in #Widget
preferable Reuse
Best in #Widget
Average in #Widget
Top functions reviewed by kandi - BETA

kandi has reviewed preferable and discovered the below as its top functions. This is intended to give you an instant insight into preferable implemented functionality, and help decide if they suit your requirements.

  • Define a default preference
  • Sets a hash of preferences .
  • Sets the preferences for this object .

preferable Key Features

User preference management for ActiveRecord

Installation

copy iconCopydownload iconDownload
$ bundle install

Usage Examples:

copy iconCopydownload iconDownload
class User < ActiveRecord::Base

  preferable do
    integer :theme_id
    boolean :newsletter, :default => false
    string  :font_color, :default => "444444"
  end

end

License

copy iconCopydownload iconDownload
Copyright (C) 2011 Dimitrij Denissenko

Pandas DataFrame update cell values from second DataFrame

copy iconCopydownload iconDownload
df1.update(df2)
   a    b  c
0  0  1.0  0
1  0  0.0  0
2  0  2.0  0
3  0  0.0  0
4  0  3.0  0
df1.update(df2.convert_dtypes())
   a  b  c
0  0  1  0
1  0  0  0
2  0  2  0
3  0  0  0
4  0  3  0
-----------------------
df1.update(df2)
   a    b  c
0  0  1.0  0
1  0  0.0  0
2  0  2.0  0
3  0  0.0  0
4  0  3.0  0
df1.update(df2.convert_dtypes())
   a  b  c
0  0  1  0
1  0  0  0
2  0  2  0
3  0  0  0
4  0  3  0
-----------------------
df1.update(df2)
   a    b  c
0  0  1.0  0
1  0  0.0  0
2  0  2.0  0
3  0  0.0  0
4  0  3.0  0
df1.update(df2.convert_dtypes())
   a  b  c
0  0  1  0
1  0  0  0
2  0  2  0
3  0  0  0
4  0  3  0
-----------------------
df1.update(df2)
   a    b  c
0  0  1.0  0
1  0  0.0  0
2  0  2.0  0
3  0  0.0  0
4  0  3.0  0
df1.update(df2.convert_dtypes())
   a  b  c
0  0  1  0
1  0  0  0
2  0  2  0
3  0  0  0
4  0  3  0

How to return const& from std::visit?

copy iconCopydownload iconDownload
return std::visit(overloaded{[](auto const& o) -> auto const & {
                               return o.get_name();
                             }}, data);

How can I get a list of all strings "greater" than a specific value in a List<string>?

copy iconCopydownload iconDownload
  var list = new List<string> {"Ringo", "John", "Paul", "George" };

  string value = "George";

  var result = list
    .Where(item => StringComparer.OrdinalIgnoreCase.Compare(item, value) > 0)
    .OrderBy(item => item)
    .ToList(); // if you want to get a list as an answer

MongoDB and LINQ: &quot;NOT IN&quot; clause

copy iconCopydownload iconDownload
                var results = capturesCollection.Aggregate()
                    .As<CaptureWithCompletions>()
                    .Lookup(processed, x => x.ImageName, x => x.ImageName, @as:(CaptureWithCompletions x) => x.CompletedCaptures)
                    .Match(x => !x.CompletedCaptures.Any())
                    //.Limit(2)
                    .ToList();

How can I extend the quantile regression lines geom_quantile to forecast in ggplot?

copy iconCopydownload iconDownload
mod  <- quantreg::rq(I(1/hwy) ~ displ, tau = c(0.25, 0.5, 0.75), data = mpg)
r_df <- setNames(as.data.frame(t(coef(mod))), c("intercept", "gradient"))

m1 + geom_abline(data = r_df, aes(slope = gradient, intercept = intercept)) +
     scale_x_continuous(limits = c(1,9),
                      breaks = waiver(),
                      n.breaks = 8)

How to find a missing row between two similar 2d numpy arrays

copy iconCopydownload iconDownload
bb_missing = aa[~np.all(aa==bb[:, None], axis=2).any(axis=0)]
>>> bb_missing
array([[ 3,  4],
       [48, 59]])
-----------------------
bb_missing = aa[~np.all(aa==bb[:, None], axis=2).any(axis=0)]
>>> bb_missing
array([[ 3,  4],
       [48, 59]])
-----------------------
aa = np.array([[1,2],[3,4],[9,10],[48,59]])
bb = np.array([[1,2],[9,10]])
idx
>>> array([ True, False,  True, False])
aa[~idx]
>>> 
array([[ 3,  4],
       [48, 59]])
def _view1D(a, b): # a, b are arrays
    a = np.ascontiguousarray(a, dtype=np.uint32)
    b = np.ascontiguousarray(b, dtype=np.uint32)
    uint32_dt = np.dtype([('', np.uint32)]*2)
    return a.view(uint32_dt).ravel(),  b.view(uint32_dt).ravel()

A, B = _view1D(aa, bb)
idx = np.isin(A, B)
def numpy_dimreduce(arr, M):
    return np.ravel_multi_index(arr.T, M, order='F')

M = aa.max(axis=0)+1
idx = np.isin(numpy_dimreduce(aa, M), numpy_dimreduce(bb, M))
@njit(parallel=True)
def _numba_dot(arr, dimshape, len_arr, len_dimshape, su):
    for i in prange(len_arr):
        for j in range(len_dimshape):
            su[i] = su[i] + arr[i][j] * dimshape[j]
        
def numba_dimreduce(arr, M, dtype=np.int64):
    '''not safe if allocation is exceeded'''
    dimshape = np.cumprod(np.insert(M[:-1], 0, 1))
    su = np.zeros(len(arr), dtype=dtype)
    _numba_dot(arr, dimshape, len(arr), len(dimshape), su)
    return su

M = aa.max(axis=0)+1
idx = np.isin(numba_dimreduce(aa, M), numba_dimreduce(bb, M))
def searchsort(A, B):
    sidx = A.argsort()
    out = np.zeros(len(A), dtype=bool)
    s = np.searchsorted(A, B, sorter=sidx)    
    out[sidx[s]] = True
    return out
def masking(A, B):
    mask = np.empty(shape=np.prod(M), dtype=bool)
    mask[A] = 0
    mask[B] = 1
    return mask[A]
def isin2D(a, b, view=_view1D, search=np.isin):
    # choose your optional view and search
    A, B = view(a), view(b)
    idx = search(A, B)
    return idx
def isin2D(a, b):
    M = a.max(axis=0) + 1
    A = np.ravel_multi_index(a.T, M, order='C')
    B = np.ravel_multi_index(b.T, M, order='C')
    idx = np.isin(A, B)
    return idx

idx = isin2D(aa, bb)
aa[~idx]
>>> 
array([[ 3,  4],
       [48, 59]])
import numpy as np
from numba import njit, prange
import benchit
%matplotlib inline
benchit.setparams(rep=3)

sizes = [100000, 300000, 500000, 1000000, 1700000, 3000000, 5000000]#, 10000000, 30000000]
N = sizes[-1]
aa = np.random.randint(0, 1000000, size=(N, 2))
r = np.random.randint(0, 2, size=N).astype(bool)
M = aa.max(axis=0) + 1

def _view1D(arr, dtype=np.uint32):
    a = np.ascontiguousarray(arr, dtype=dtype)
    dt = np.dtype([('', dtype)]*2)
    return a.view(dt).ravel()

def _numpy_dimreduce(arr, M):
    return np.ravel_multi_index(arr.T, M, order='C')

@njit(parallel=True)
def _numba_dot(arr, dimshape, len_arr, len_dimshape, su):
    for i in prange(len_arr):
        for j in range(len_dimshape):
            su[i] = su[i] + arr[i][j] * dimshape[j]
        
def _numba_dimreduce(arr, M, dtype=np.int64):
    '''not safe if allocation is exceeded'''
    dimshape = np.cumprod(np.insert(M[:-1], 0, 1))
    su = np.zeros(len(arr), dtype=dtype)
    _numba_dot(arr, dimshape, len(arr), len(dimshape), su)
    return su


def isin2D(a, b, view=_view1D, search=np.isin):
    A, B = view(a), view(b)
    idx = search(A, B)
    return idx

def searchsort(A, B):
    sidx = A.argsort()
    out = np.zeros(len(A), dtype=bool)
    s = np.searchsorted(A, B, sorter=sidx)    
    out[sidx[s]] = True
    return out

def isin2D_numpy_view(a, b): return isin2D(a, b, view=_view1D, search=np.isin)

def isin2D_numpy_dimr_isin(a, b, M=M): return isin2D(a, b, view=lambda arr, M=M: _numpy_dimreduce(arr, M), search=np.isin)
def isin2D_numba_dimr_isin(a, b, M=M): return isin2D(a, b, view=lambda arr, M=M: _numba_dimreduce(arr, M), search=np.isin)

def isin2D_numpy_dimr_searchsort(a, b, M=M): return isin2D(a, b, view=lambda arr, M=M: _numpy_dimreduce(arr, M), search=searchsort)
def isin2D_numba_dimr_searchsort(a, b, M=M): return isin2D(a, b, view=lambda arr, M=M: _numba_dimreduce(arr[:, ::-1], M[::-1]), search=searchsort)

def isin2D_numpy_dimr_in1D(a, b, M=M): return isin2D(a, b, view=lambda arr, M=M: _numpy_dimreduce(arr, M), search=np.in1d)
def isin2D_numba_dimr_in1D(a, b, M=M): return isin2D(a, b, view=lambda arr, M=M: _numba_dimreduce(arr[:, ::-1], M[::-1]), search=np.in1d)

fns = [isin2D_numpy_dimr, isin2D_numba_dimr, isin2D_numpy_view, isin2D_numpy_dimr_searchsort, isin2D_numba_dimr_searchsort, isin2D_numpy_dimr_in1D, isin2D_numba_dimr_in1D]

in_ = {s/1000000: (aa[:s], aa[:s][r[:s]]) for s in sizes}
t = benchit.timings(fns, in_, multivar=True, input_name='Millions of events')
t.plot(logx=True, figsize=(12, 6), fontsize=14)
def masking1D(a, b):
    M = aa.max(axis=0) + 1
    A = np.ravel_multi_index(a.T, M, order='C')
    B = np.ravel_multi_index(b.T, M, order='C')
    mask = np.empty(shape=np.prod(M), dtype=bool)
    mask[A] = 1
    mask[B] = 0
    return mask[A]

def masking2D(a, b):
    M = a.max(axis=0) + 1
    mask = np.empty(shape=M, dtype=bool)
    mask[tuple(a.T)] = 1
    mask[tuple(b.T)] = 0
    return mask[tuple(a.T)]

aa = np.array([[988, 986], [299, 327], [756, 30], [327, 189], [988, 294], [64, 232], [11, 46], [38, 223], [933, 770], [141, 141]])
bb = aa[[1, 2, 3, 6, 7, 4]]
masking1D(aa, bb)
>>> array([ True, False, False, False, False,  True, False, False,  True, True])
masking2D(aa, bb)
>>> array([ True, False, False, False, False,  True, False, False,  True, True])
-----------------------
aa = np.array([[1,2],[3,4],[9,10],[48,59]])
bb = np.array([[1,2],[9,10]])
idx
>>> array([ True, False,  True, False])
aa[~idx]
>>> 
array([[ 3,  4],
       [48, 59]])
def _view1D(a, b): # a, b are arrays
    a = np.ascontiguousarray(a, dtype=np.uint32)
    b = np.ascontiguousarray(b, dtype=np.uint32)
    uint32_dt = np.dtype([('', np.uint32)]*2)
    return a.view(uint32_dt).ravel(),  b.view(uint32_dt).ravel()

A, B = _view1D(aa, bb)
idx = np.isin(A, B)
def numpy_dimreduce(arr, M):
    return np.ravel_multi_index(arr.T, M, order='F')

M = aa.max(axis=0)+1
idx = np.isin(numpy_dimreduce(aa, M), numpy_dimreduce(bb, M))
@njit(parallel=True)
def _numba_dot(arr, dimshape, len_arr, len_dimshape, su):
    for i in prange(len_arr):
        for j in range(len_dimshape):
            su[i] = su[i] + arr[i][j] * dimshape[j]
        
def numba_dimreduce(arr, M, dtype=np.int64):
    '''not safe if allocation is exceeded'''
    dimshape = np.cumprod(np.insert(M[:-1], 0, 1))
    su = np.zeros(len(arr), dtype=dtype)
    _numba_dot(arr, dimshape, len(arr), len(dimshape), su)
    return su

M = aa.max(axis=0)+1
idx = np.isin(numba_dimreduce(aa, M), numba_dimreduce(bb, M))
def searchsort(A, B):
    sidx = A.argsort()
    out = np.zeros(len(A), dtype=bool)
    s = np.searchsorted(A, B, sorter=sidx)    
    out[sidx[s]] = True
    return out
def masking(A, B):
    mask = np.empty(shape=np.prod(M), dtype=bool)
    mask[A] = 0
    mask[B] = 1
    return mask[A]
def isin2D(a, b, view=_view1D, search=np.isin):
    # choose your optional view and search
    A, B = view(a), view(b)
    idx = search(A, B)
    return idx
def isin2D(a, b):
    M = a.max(axis=0) + 1
    A = np.ravel_multi_index(a.T, M, order='C')
    B = np.ravel_multi_index(b.T, M, order='C')
    idx = np.isin(A, B)
    return idx

idx = isin2D(aa, bb)
aa[~idx]
>>> 
array([[ 3,  4],
       [48, 59]])
import numpy as np
from numba import njit, prange
import benchit
%matplotlib inline
benchit.setparams(rep=3)

sizes = [100000, 300000, 500000, 1000000, 1700000, 3000000, 5000000]#, 10000000, 30000000]
N = sizes[-1]
aa = np.random.randint(0, 1000000, size=(N, 2))
r = np.random.randint(0, 2, size=N).astype(bool)
M = aa.max(axis=0) + 1

def _view1D(arr, dtype=np.uint32):
    a = np.ascontiguousarray(arr, dtype=dtype)
    dt = np.dtype([('', dtype)]*2)
    return a.view(dt).ravel()

def _numpy_dimreduce(arr, M):
    return np.ravel_multi_index(arr.T, M, order='C')

@njit(parallel=True)
def _numba_dot(arr, dimshape, len_arr, len_dimshape, su):
    for i in prange(len_arr):
        for j in range(len_dimshape):
            su[i] = su[i] + arr[i][j] * dimshape[j]
        
def _numba_dimreduce(arr, M, dtype=np.int64):
    '''not safe if allocation is exceeded'''
    dimshape = np.cumprod(np.insert(M[:-1], 0, 1))
    su = np.zeros(len(arr), dtype=dtype)
    _numba_dot(arr, dimshape, len(arr), len(dimshape), su)
    return su


def isin2D(a, b, view=_view1D, search=np.isin):
    A, B = view(a), view(b)
    idx = search(A, B)
    return idx

def searchsort(A, B):
    sidx = A.argsort()
    out = np.zeros(len(A), dtype=bool)
    s = np.searchsorted(A, B, sorter=sidx)    
    out[sidx[s]] = True
    return out

def isin2D_numpy_view(a, b): return isin2D(a, b, view=_view1D, search=np.isin)

def isin2D_numpy_dimr_isin(a, b, M=M): return isin2D(a, b, view=lambda arr, M=M: _numpy_dimreduce(arr, M), search=np.isin)
def isin2D_numba_dimr_isin(a, b, M=M): return isin2D(a, b, view=lambda arr, M=M: _numba_dimreduce(arr, M), search=np.isin)

def isin2D_numpy_dimr_searchsort(a, b, M=M): return isin2D(a, b, view=lambda arr, M=M: _numpy_dimreduce(arr, M), search=searchsort)
def isin2D_numba_dimr_searchsort(a, b, M=M): return isin2D(a, b, view=lambda arr, M=M: _numba_dimreduce(arr[:, ::-1], M[::-1]), search=searchsort)

def isin2D_numpy_dimr_in1D(a, b, M=M): return isin2D(a, b, view=lambda arr, M=M: _numpy_dimreduce(arr, M), search=np.in1d)
def isin2D_numba_dimr_in1D(a, b, M=M): return isin2D(a, b, view=lambda arr, M=M: _numba_dimreduce(arr[:, ::-1], M[::-1]), search=np.in1d)

fns = [isin2D_numpy_dimr, isin2D_numba_dimr, isin2D_numpy_view, isin2D_numpy_dimr_searchsort, isin2D_numba_dimr_searchsort, isin2D_numpy_dimr_in1D, isin2D_numba_dimr_in1D]

in_ = {s/1000000: (aa[:s], aa[:s][r[:s]]) for s in sizes}
t = benchit.timings(fns, in_, multivar=True, input_name='Millions of events')
t.plot(logx=True, figsize=(12, 6), fontsize=14)
def masking1D(a, b):
    M = aa.max(axis=0) + 1
    A = np.ravel_multi_index(a.T, M, order='C')
    B = np.ravel_multi_index(b.T, M, order='C')
    mask = np.empty(shape=np.prod(M), dtype=bool)
    mask[A] = 1
    mask[B] = 0
    return mask[A]

def masking2D(a, b):
    M = a.max(axis=0) + 1
    mask = np.empty(shape=M, dtype=bool)
    mask[tuple(a.T)] = 1
    mask[tuple(b.T)] = 0
    return mask[tuple(a.T)]

aa = np.array([[988, 986], [299, 327], [756, 30], [327, 189], [988, 294], [64, 232], [11, 46], [38, 223], [933, 770], [141, 141]])
bb = aa[[1, 2, 3, 6, 7, 4]]
masking1D(aa, bb)
>>> array([ True, False, False, False, False,  True, False, False,  True, True])
masking2D(aa, bb)
>>> array([ True, False, False, False, False,  True, False, False,  True, True])
-----------------------
aa = np.array([[1,2],[3,4],[9,10],[48,59]])
bb = np.array([[1,2],[9,10]])
idx
>>> array([ True, False,  True, False])
aa[~idx]
>>> 
array([[ 3,  4],
       [48, 59]])
def _view1D(a, b): # a, b are arrays
    a = np.ascontiguousarray(a, dtype=np.uint32)
    b = np.ascontiguousarray(b, dtype=np.uint32)
    uint32_dt = np.dtype([('', np.uint32)]*2)
    return a.view(uint32_dt).ravel(),  b.view(uint32_dt).ravel()

A, B = _view1D(aa, bb)
idx = np.isin(A, B)
def numpy_dimreduce(arr, M):
    return np.ravel_multi_index(arr.T, M, order='F')

M = aa.max(axis=0)+1
idx = np.isin(numpy_dimreduce(aa, M), numpy_dimreduce(bb, M))
@njit(parallel=True)
def _numba_dot(arr, dimshape, len_arr, len_dimshape, su):
    for i in prange(len_arr):
        for j in range(len_dimshape):
            su[i] = su[i] + arr[i][j] * dimshape[j]
        
def numba_dimreduce(arr, M, dtype=np.int64):
    '''not safe if allocation is exceeded'''
    dimshape = np.cumprod(np.insert(M[:-1], 0, 1))
    su = np.zeros(len(arr), dtype=dtype)
    _numba_dot(arr, dimshape, len(arr), len(dimshape), su)
    return su

M = aa.max(axis=0)+1
idx = np.isin(numba_dimreduce(aa, M), numba_dimreduce(bb, M))
def searchsort(A, B):
    sidx = A.argsort()
    out = np.zeros(len(A), dtype=bool)
    s = np.searchsorted(A, B, sorter=sidx)    
    out[sidx[s]] = True
    return out
def masking(A, B):
    mask = np.empty(shape=np.prod(M), dtype=bool)
    mask[A] = 0
    mask[B] = 1
    return mask[A]
def isin2D(a, b, view=_view1D, search=np.isin):
    # choose your optional view and search
    A, B = view(a), view(b)
    idx = search(A, B)
    return idx
def isin2D(a, b):
    M = a.max(axis=0) + 1
    A = np.ravel_multi_index(a.T, M, order='C')
    B = np.ravel_multi_index(b.T, M, order='C')
    idx = np.isin(A, B)
    return idx

idx = isin2D(aa, bb)
aa[~idx]
>>> 
array([[ 3,  4],
       [48, 59]])
import numpy as np
from numba import njit, prange
import benchit
%matplotlib inline
benchit.setparams(rep=3)

sizes = [100000, 300000, 500000, 1000000, 1700000, 3000000, 5000000]#, 10000000, 30000000]
N = sizes[-1]
aa = np.random.randint(0, 1000000, size=(N, 2))
r = np.random.randint(0, 2, size=N).astype(bool)
M = aa.max(axis=0) + 1

def _view1D(arr, dtype=np.uint32):
    a = np.ascontiguousarray(arr, dtype=dtype)
    dt = np.dtype([('', dtype)]*2)
    return a.view(dt).ravel()

def _numpy_dimreduce(arr, M):
    return np.ravel_multi_index(arr.T, M, order='C')

@njit(parallel=True)
def _numba_dot(arr, dimshape, len_arr, len_dimshape, su):
    for i in prange(len_arr):
        for j in range(len_dimshape):
            su[i] = su[i] + arr[i][j] * dimshape[j]
        
def _numba_dimreduce(arr, M, dtype=np.int64):
    '''not safe if allocation is exceeded'''
    dimshape = np.cumprod(np.insert(M[:-1], 0, 1))
    su = np.zeros(len(arr), dtype=dtype)
    _numba_dot(arr, dimshape, len(arr), len(dimshape), su)
    return su


def isin2D(a, b, view=_view1D, search=np.isin):
    A, B = view(a), view(b)
    idx = search(A, B)
    return idx

def searchsort(A, B):
    sidx = A.argsort()
    out = np.zeros(len(A), dtype=bool)
    s = np.searchsorted(A, B, sorter=sidx)    
    out[sidx[s]] = True
    return out

def isin2D_numpy_view(a, b): return isin2D(a, b, view=_view1D, search=np.isin)

def isin2D_numpy_dimr_isin(a, b, M=M): return isin2D(a, b, view=lambda arr, M=M: _numpy_dimreduce(arr, M), search=np.isin)
def isin2D_numba_dimr_isin(a, b, M=M): return isin2D(a, b, view=lambda arr, M=M: _numba_dimreduce(arr, M), search=np.isin)

def isin2D_numpy_dimr_searchsort(a, b, M=M): return isin2D(a, b, view=lambda arr, M=M: _numpy_dimreduce(arr, M), search=searchsort)
def isin2D_numba_dimr_searchsort(a, b, M=M): return isin2D(a, b, view=lambda arr, M=M: _numba_dimreduce(arr[:, ::-1], M[::-1]), search=searchsort)

def isin2D_numpy_dimr_in1D(a, b, M=M): return isin2D(a, b, view=lambda arr, M=M: _numpy_dimreduce(arr, M), search=np.in1d)
def isin2D_numba_dimr_in1D(a, b, M=M): return isin2D(a, b, view=lambda arr, M=M: _numba_dimreduce(arr[:, ::-1], M[::-1]), search=np.in1d)

fns = [isin2D_numpy_dimr, isin2D_numba_dimr, isin2D_numpy_view, isin2D_numpy_dimr_searchsort, isin2D_numba_dimr_searchsort, isin2D_numpy_dimr_in1D, isin2D_numba_dimr_in1D]

in_ = {s/1000000: (aa[:s], aa[:s][r[:s]]) for s in sizes}
t = benchit.timings(fns, in_, multivar=True, input_name='Millions of events')
t.plot(logx=True, figsize=(12, 6), fontsize=14)
def masking1D(a, b):
    M = aa.max(axis=0) + 1
    A = np.ravel_multi_index(a.T, M, order='C')
    B = np.ravel_multi_index(b.T, M, order='C')
    mask = np.empty(shape=np.prod(M), dtype=bool)
    mask[A] = 1
    mask[B] = 0
    return mask[A]

def masking2D(a, b):
    M = a.max(axis=0) + 1
    mask = np.empty(shape=M, dtype=bool)
    mask[tuple(a.T)] = 1
    mask[tuple(b.T)] = 0
    return mask[tuple(a.T)]

aa = np.array([[988, 986], [299, 327], [756, 30], [327, 189], [988, 294], [64, 232], [11, 46], [38, 223], [933, 770], [141, 141]])
bb = aa[[1, 2, 3, 6, 7, 4]]
masking1D(aa, bb)
>>> array([ True, False, False, False, False,  True, False, False,  True, True])
masking2D(aa, bb)
>>> array([ True, False, False, False, False,  True, False, False,  True, True])
-----------------------
aa = np.array([[1,2],[3,4],[9,10],[48,59]])
bb = np.array([[1,2],[9,10]])
idx
>>> array([ True, False,  True, False])
aa[~idx]
>>> 
array([[ 3,  4],
       [48, 59]])
def _view1D(a, b): # a, b are arrays
    a = np.ascontiguousarray(a, dtype=np.uint32)
    b = np.ascontiguousarray(b, dtype=np.uint32)
    uint32_dt = np.dtype([('', np.uint32)]*2)
    return a.view(uint32_dt).ravel(),  b.view(uint32_dt).ravel()

A, B = _view1D(aa, bb)
idx = np.isin(A, B)
def numpy_dimreduce(arr, M):
    return np.ravel_multi_index(arr.T, M, order='F')

M = aa.max(axis=0)+1
idx = np.isin(numpy_dimreduce(aa, M), numpy_dimreduce(bb, M))
@njit(parallel=True)
def _numba_dot(arr, dimshape, len_arr, len_dimshape, su):
    for i in prange(len_arr):
        for j in range(len_dimshape):
            su[i] = su[i] + arr[i][j] * dimshape[j]
        
def numba_dimreduce(arr, M, dtype=np.int64):
    '''not safe if allocation is exceeded'''
    dimshape = np.cumprod(np.insert(M[:-1], 0, 1))
    su = np.zeros(len(arr), dtype=dtype)
    _numba_dot(arr, dimshape, len(arr), len(dimshape), su)
    return su

M = aa.max(axis=0)+1
idx = np.isin(numba_dimreduce(aa, M), numba_dimreduce(bb, M))
def searchsort(A, B):
    sidx = A.argsort()
    out = np.zeros(len(A), dtype=bool)
    s = np.searchsorted(A, B, sorter=sidx)    
    out[sidx[s]] = True
    return out
def masking(A, B):
    mask = np.empty(shape=np.prod(M), dtype=bool)
    mask[A] = 0
    mask[B] = 1
    return mask[A]
def isin2D(a, b, view=_view1D, search=np.isin):
    # choose your optional view and search
    A, B = view(a), view(b)
    idx = search(A, B)
    return idx
def isin2D(a, b):
    M = a.max(axis=0) + 1
    A = np.ravel_multi_index(a.T, M, order='C')
    B = np.ravel_multi_index(b.T, M, order='C')
    idx = np.isin(A, B)
    return idx

idx = isin2D(aa, bb)
aa[~idx]
>>> 
array([[ 3,  4],
       [48, 59]])
import numpy as np
from numba import njit, prange
import benchit
%matplotlib inline
benchit.setparams(rep=3)

sizes = [100000, 300000, 500000, 1000000, 1700000, 3000000, 5000000]#, 10000000, 30000000]
N = sizes[-1]
aa = np.random.randint(0, 1000000, size=(N, 2))
r = np.random.randint(0, 2, size=N).astype(bool)
M = aa.max(axis=0) + 1

def _view1D(arr, dtype=np.uint32):
    a = np.ascontiguousarray(arr, dtype=dtype)
    dt = np.dtype([('', dtype)]*2)
    return a.view(dt).ravel()

def _numpy_dimreduce(arr, M):
    return np.ravel_multi_index(arr.T, M, order='C')

@njit(parallel=True)
def _numba_dot(arr, dimshape, len_arr, len_dimshape, su):
    for i in prange(len_arr):
        for j in range(len_dimshape):
            su[i] = su[i] + arr[i][j] * dimshape[j]
        
def _numba_dimreduce(arr, M, dtype=np.int64):
    '''not safe if allocation is exceeded'''
    dimshape = np.cumprod(np.insert(M[:-1], 0, 1))
    su = np.zeros(len(arr), dtype=dtype)
    _numba_dot(arr, dimshape, len(arr), len(dimshape), su)
    return su


def isin2D(a, b, view=_view1D, search=np.isin):
    A, B = view(a), view(b)
    idx = search(A, B)
    return idx

def searchsort(A, B):
    sidx = A.argsort()
    out = np.zeros(len(A), dtype=bool)
    s = np.searchsorted(A, B, sorter=sidx)    
    out[sidx[s]] = True
    return out

def isin2D_numpy_view(a, b): return isin2D(a, b, view=_view1D, search=np.isin)

def isin2D_numpy_dimr_isin(a, b, M=M): return isin2D(a, b, view=lambda arr, M=M: _numpy_dimreduce(arr, M), search=np.isin)
def isin2D_numba_dimr_isin(a, b, M=M): return isin2D(a, b, view=lambda arr, M=M: _numba_dimreduce(arr, M), search=np.isin)

def isin2D_numpy_dimr_searchsort(a, b, M=M): return isin2D(a, b, view=lambda arr, M=M: _numpy_dimreduce(arr, M), search=searchsort)
def isin2D_numba_dimr_searchsort(a, b, M=M): return isin2D(a, b, view=lambda arr, M=M: _numba_dimreduce(arr[:, ::-1], M[::-1]), search=searchsort)

def isin2D_numpy_dimr_in1D(a, b, M=M): return isin2D(a, b, view=lambda arr, M=M: _numpy_dimreduce(arr, M), search=np.in1d)
def isin2D_numba_dimr_in1D(a, b, M=M): return isin2D(a, b, view=lambda arr, M=M: _numba_dimreduce(arr[:, ::-1], M[::-1]), search=np.in1d)

fns = [isin2D_numpy_dimr, isin2D_numba_dimr, isin2D_numpy_view, isin2D_numpy_dimr_searchsort, isin2D_numba_dimr_searchsort, isin2D_numpy_dimr_in1D, isin2D_numba_dimr_in1D]

in_ = {s/1000000: (aa[:s], aa[:s][r[:s]]) for s in sizes}
t = benchit.timings(fns, in_, multivar=True, input_name='Millions of events')
t.plot(logx=True, figsize=(12, 6), fontsize=14)
def masking1D(a, b):
    M = aa.max(axis=0) + 1
    A = np.ravel_multi_index(a.T, M, order='C')
    B = np.ravel_multi_index(b.T, M, order='C')
    mask = np.empty(shape=np.prod(M), dtype=bool)
    mask[A] = 1
    mask[B] = 0
    return mask[A]

def masking2D(a, b):
    M = a.max(axis=0) + 1
    mask = np.empty(shape=M, dtype=bool)
    mask[tuple(a.T)] = 1
    mask[tuple(b.T)] = 0
    return mask[tuple(a.T)]

aa = np.array([[988, 986], [299, 327], [756, 30], [327, 189], [988, 294], [64, 232], [11, 46], [38, 223], [933, 770], [141, 141]])
bb = aa[[1, 2, 3, 6, 7, 4]]
masking1D(aa, bb)
>>> array([ True, False, False, False, False,  True, False, False,  True, True])
masking2D(aa, bb)
>>> array([ True, False, False, False, False,  True, False, False,  True, True])
-----------------------
aa = np.array([[1,2],[3,4],[9,10],[48,59]])
bb = np.array([[1,2],[9,10]])
idx
>>> array([ True, False,  True, False])
aa[~idx]
>>> 
array([[ 3,  4],
       [48, 59]])
def _view1D(a, b): # a, b are arrays
    a = np.ascontiguousarray(a, dtype=np.uint32)
    b = np.ascontiguousarray(b, dtype=np.uint32)
    uint32_dt = np.dtype([('', np.uint32)]*2)
    return a.view(uint32_dt).ravel(),  b.view(uint32_dt).ravel()

A, B = _view1D(aa, bb)
idx = np.isin(A, B)
def numpy_dimreduce(arr, M):
    return np.ravel_multi_index(arr.T, M, order='F')

M = aa.max(axis=0)+1
idx = np.isin(numpy_dimreduce(aa, M), numpy_dimreduce(bb, M))
@njit(parallel=True)
def _numba_dot(arr, dimshape, len_arr, len_dimshape, su):
    for i in prange(len_arr):
        for j in range(len_dimshape):
            su[i] = su[i] + arr[i][j] * dimshape[j]
        
def numba_dimreduce(arr, M, dtype=np.int64):
    '''not safe if allocation is exceeded'''
    dimshape = np.cumprod(np.insert(M[:-1], 0, 1))
    su = np.zeros(len(arr), dtype=dtype)
    _numba_dot(arr, dimshape, len(arr), len(dimshape), su)
    return su

M = aa.max(axis=0)+1
idx = np.isin(numba_dimreduce(aa, M), numba_dimreduce(bb, M))
def searchsort(A, B):
    sidx = A.argsort()
    out = np.zeros(len(A), dtype=bool)
    s = np.searchsorted(A, B, sorter=sidx)    
    out[sidx[s]] = True
    return out
def masking(A, B):
    mask = np.empty(shape=np.prod(M), dtype=bool)
    mask[A] = 0
    mask[B] = 1
    return mask[A]
def isin2D(a, b, view=_view1D, search=np.isin):
    # choose your optional view and search
    A, B = view(a), view(b)
    idx = search(A, B)
    return idx
def isin2D(a, b):
    M = a.max(axis=0) + 1
    A = np.ravel_multi_index(a.T, M, order='C')
    B = np.ravel_multi_index(b.T, M, order='C')
    idx = np.isin(A, B)
    return idx

idx = isin2D(aa, bb)
aa[~idx]
>>> 
array([[ 3,  4],
       [48, 59]])
import numpy as np
from numba import njit, prange
import benchit
%matplotlib inline
benchit.setparams(rep=3)

sizes = [100000, 300000, 500000, 1000000, 1700000, 3000000, 5000000]#, 10000000, 30000000]
N = sizes[-1]
aa = np.random.randint(0, 1000000, size=(N, 2))
r = np.random.randint(0, 2, size=N).astype(bool)
M = aa.max(axis=0) + 1

def _view1D(arr, dtype=np.uint32):
    a = np.ascontiguousarray(arr, dtype=dtype)
    dt = np.dtype([('', dtype)]*2)
    return a.view(dt).ravel()

def _numpy_dimreduce(arr, M):
    return np.ravel_multi_index(arr.T, M, order='C')

@njit(parallel=True)
def _numba_dot(arr, dimshape, len_arr, len_dimshape, su):
    for i in prange(len_arr):
        for j in range(len_dimshape):
            su[i] = su[i] + arr[i][j] * dimshape[j]
        
def _numba_dimreduce(arr, M, dtype=np.int64):
    '''not safe if allocation is exceeded'''
    dimshape = np.cumprod(np.insert(M[:-1], 0, 1))
    su = np.zeros(len(arr), dtype=dtype)
    _numba_dot(arr, dimshape, len(arr), len(dimshape), su)
    return su


def isin2D(a, b, view=_view1D, search=np.isin):
    A, B = view(a), view(b)
    idx = search(A, B)
    return idx

def searchsort(A, B):
    sidx = A.argsort()
    out = np.zeros(len(A), dtype=bool)
    s = np.searchsorted(A, B, sorter=sidx)    
    out[sidx[s]] = True
    return out

def isin2D_numpy_view(a, b): return isin2D(a, b, view=_view1D, search=np.isin)

def isin2D_numpy_dimr_isin(a, b, M=M): return isin2D(a, b, view=lambda arr, M=M: _numpy_dimreduce(arr, M), search=np.isin)
def isin2D_numba_dimr_isin(a, b, M=M): return isin2D(a, b, view=lambda arr, M=M: _numba_dimreduce(arr, M), search=np.isin)

def isin2D_numpy_dimr_searchsort(a, b, M=M): return isin2D(a, b, view=lambda arr, M=M: _numpy_dimreduce(arr, M), search=searchsort)
def isin2D_numba_dimr_searchsort(a, b, M=M): return isin2D(a, b, view=lambda arr, M=M: _numba_dimreduce(arr[:, ::-1], M[::-1]), search=searchsort)

def isin2D_numpy_dimr_in1D(a, b, M=M): return isin2D(a, b, view=lambda arr, M=M: _numpy_dimreduce(arr, M), search=np.in1d)
def isin2D_numba_dimr_in1D(a, b, M=M): return isin2D(a, b, view=lambda arr, M=M: _numba_dimreduce(arr[:, ::-1], M[::-1]), search=np.in1d)

fns = [isin2D_numpy_dimr, isin2D_numba_dimr, isin2D_numpy_view, isin2D_numpy_dimr_searchsort, isin2D_numba_dimr_searchsort, isin2D_numpy_dimr_in1D, isin2D_numba_dimr_in1D]

in_ = {s/1000000: (aa[:s], aa[:s][r[:s]]) for s in sizes}
t = benchit.timings(fns, in_, multivar=True, input_name='Millions of events')
t.plot(logx=True, figsize=(12, 6), fontsize=14)
def masking1D(a, b):
    M = aa.max(axis=0) + 1
    A = np.ravel_multi_index(a.T, M, order='C')
    B = np.ravel_multi_index(b.T, M, order='C')
    mask = np.empty(shape=np.prod(M), dtype=bool)
    mask[A] = 1
    mask[B] = 0
    return mask[A]

def masking2D(a, b):
    M = a.max(axis=0) + 1
    mask = np.empty(shape=M, dtype=bool)
    mask[tuple(a.T)] = 1
    mask[tuple(b.T)] = 0
    return mask[tuple(a.T)]

aa = np.array([[988, 986], [299, 327], [756, 30], [327, 189], [988, 294], [64, 232], [11, 46], [38, 223], [933, 770], [141, 141]])
bb = aa[[1, 2, 3, 6, 7, 4]]
masking1D(aa, bb)
>>> array([ True, False, False, False, False,  True, False, False,  True, True])
masking2D(aa, bb)
>>> array([ True, False, False, False, False,  True, False, False,  True, True])
-----------------------
aa = np.array([[1,2],[3,4],[9,10],[48,59]])
bb = np.array([[1,2],[9,10]])
idx
>>> array([ True, False,  True, False])
aa[~idx]
>>> 
array([[ 3,  4],
       [48, 59]])
def _view1D(a, b): # a, b are arrays
    a = np.ascontiguousarray(a, dtype=np.uint32)
    b = np.ascontiguousarray(b, dtype=np.uint32)
    uint32_dt = np.dtype([('', np.uint32)]*2)
    return a.view(uint32_dt).ravel(),  b.view(uint32_dt).ravel()

A, B = _view1D(aa, bb)
idx = np.isin(A, B)
def numpy_dimreduce(arr, M):
    return np.ravel_multi_index(arr.T, M, order='F')

M = aa.max(axis=0)+1
idx = np.isin(numpy_dimreduce(aa, M), numpy_dimreduce(bb, M))
@njit(parallel=True)
def _numba_dot(arr, dimshape, len_arr, len_dimshape, su):
    for i in prange(len_arr):
        for j in range(len_dimshape):
            su[i] = su[i] + arr[i][j] * dimshape[j]
        
def numba_dimreduce(arr, M, dtype=np.int64):
    '''not safe if allocation is exceeded'''
    dimshape = np.cumprod(np.insert(M[:-1], 0, 1))
    su = np.zeros(len(arr), dtype=dtype)
    _numba_dot(arr, dimshape, len(arr), len(dimshape), su)
    return su

M = aa.max(axis=0)+1
idx = np.isin(numba_dimreduce(aa, M), numba_dimreduce(bb, M))
def searchsort(A, B):
    sidx = A.argsort()
    out = np.zeros(len(A), dtype=bool)
    s = np.searchsorted(A, B, sorter=sidx)    
    out[sidx[s]] = True
    return out
def masking(A, B):
    mask = np.empty(shape=np.prod(M), dtype=bool)
    mask[A] = 0
    mask[B] = 1
    return mask[A]
def isin2D(a, b, view=_view1D, search=np.isin):
    # choose your optional view and search
    A, B = view(a), view(b)
    idx = search(A, B)
    return idx
def isin2D(a, b):
    M = a.max(axis=0) + 1
    A = np.ravel_multi_index(a.T, M, order='C')
    B = np.ravel_multi_index(b.T, M, order='C')
    idx = np.isin(A, B)
    return idx

idx = isin2D(aa, bb)
aa[~idx]
>>> 
array([[ 3,  4],
       [48, 59]])
import numpy as np
from numba import njit, prange
import benchit
%matplotlib inline
benchit.setparams(rep=3)

sizes = [100000, 300000, 500000, 1000000, 1700000, 3000000, 5000000]#, 10000000, 30000000]
N = sizes[-1]
aa = np.random.randint(0, 1000000, size=(N, 2))
r = np.random.randint(0, 2, size=N).astype(bool)
M = aa.max(axis=0) + 1

def _view1D(arr, dtype=np.uint32):
    a = np.ascontiguousarray(arr, dtype=dtype)
    dt = np.dtype([('', dtype)]*2)
    return a.view(dt).ravel()

def _numpy_dimreduce(arr, M):
    return np.ravel_multi_index(arr.T, M, order='C')

@njit(parallel=True)
def _numba_dot(arr, dimshape, len_arr, len_dimshape, su):
    for i in prange(len_arr):
        for j in range(len_dimshape):
            su[i] = su[i] + arr[i][j] * dimshape[j]
        
def _numba_dimreduce(arr, M, dtype=np.int64):
    '''not safe if allocation is exceeded'''
    dimshape = np.cumprod(np.insert(M[:-1], 0, 1))
    su = np.zeros(len(arr), dtype=dtype)
    _numba_dot(arr, dimshape, len(arr), len(dimshape), su)
    return su


def isin2D(a, b, view=_view1D, search=np.isin):
    A, B = view(a), view(b)
    idx = search(A, B)
    return idx

def searchsort(A, B):
    sidx = A.argsort()
    out = np.zeros(len(A), dtype=bool)
    s = np.searchsorted(A, B, sorter=sidx)    
    out[sidx[s]] = True
    return out

def isin2D_numpy_view(a, b): return isin2D(a, b, view=_view1D, search=np.isin)

def isin2D_numpy_dimr_isin(a, b, M=M): return isin2D(a, b, view=lambda arr, M=M: _numpy_dimreduce(arr, M), search=np.isin)
def isin2D_numba_dimr_isin(a, b, M=M): return isin2D(a, b, view=lambda arr, M=M: _numba_dimreduce(arr, M), search=np.isin)

def isin2D_numpy_dimr_searchsort(a, b, M=M): return isin2D(a, b, view=lambda arr, M=M: _numpy_dimreduce(arr, M), search=searchsort)
def isin2D_numba_dimr_searchsort(a, b, M=M): return isin2D(a, b, view=lambda arr, M=M: _numba_dimreduce(arr[:, ::-1], M[::-1]), search=searchsort)

def isin2D_numpy_dimr_in1D(a, b, M=M): return isin2D(a, b, view=lambda arr, M=M: _numpy_dimreduce(arr, M), search=np.in1d)
def isin2D_numba_dimr_in1D(a, b, M=M): return isin2D(a, b, view=lambda arr, M=M: _numba_dimreduce(arr[:, ::-1], M[::-1]), search=np.in1d)

fns = [isin2D_numpy_dimr, isin2D_numba_dimr, isin2D_numpy_view, isin2D_numpy_dimr_searchsort, isin2D_numba_dimr_searchsort, isin2D_numpy_dimr_in1D, isin2D_numba_dimr_in1D]

in_ = {s/1000000: (aa[:s], aa[:s][r[:s]]) for s in sizes}
t = benchit.timings(fns, in_, multivar=True, input_name='Millions of events')
t.plot(logx=True, figsize=(12, 6), fontsize=14)
def masking1D(a, b):
    M = aa.max(axis=0) + 1
    A = np.ravel_multi_index(a.T, M, order='C')
    B = np.ravel_multi_index(b.T, M, order='C')
    mask = np.empty(shape=np.prod(M), dtype=bool)
    mask[A] = 1
    mask[B] = 0
    return mask[A]

def masking2D(a, b):
    M = a.max(axis=0) + 1
    mask = np.empty(shape=M, dtype=bool)
    mask[tuple(a.T)] = 1
    mask[tuple(b.T)] = 0
    return mask[tuple(a.T)]

aa = np.array([[988, 986], [299, 327], [756, 30], [327, 189], [988, 294], [64, 232], [11, 46], [38, 223], [933, 770], [141, 141]])
bb = aa[[1, 2, 3, 6, 7, 4]]
masking1D(aa, bb)
>>> array([ True, False, False, False, False,  True, False, False,  True, True])
masking2D(aa, bb)
>>> array([ True, False, False, False, False,  True, False, False,  True, True])
-----------------------
aa = np.array([[1,2],[3,4],[9,10],[48,59]])
bb = np.array([[1,2],[9,10]])
idx
>>> array([ True, False,  True, False])
aa[~idx]
>>> 
array([[ 3,  4],
       [48, 59]])
def _view1D(a, b): # a, b are arrays
    a = np.ascontiguousarray(a, dtype=np.uint32)
    b = np.ascontiguousarray(b, dtype=np.uint32)
    uint32_dt = np.dtype([('', np.uint32)]*2)
    return a.view(uint32_dt).ravel(),  b.view(uint32_dt).ravel()

A, B = _view1D(aa, bb)
idx = np.isin(A, B)
def numpy_dimreduce(arr, M):
    return np.ravel_multi_index(arr.T, M, order='F')

M = aa.max(axis=0)+1
idx = np.isin(numpy_dimreduce(aa, M), numpy_dimreduce(bb, M))
@njit(parallel=True)
def _numba_dot(arr, dimshape, len_arr, len_dimshape, su):
    for i in prange(len_arr):
        for j in range(len_dimshape):
            su[i] = su[i] + arr[i][j] * dimshape[j]
        
def numba_dimreduce(arr, M, dtype=np.int64):
    '''not safe if allocation is exceeded'''
    dimshape = np.cumprod(np.insert(M[:-1], 0, 1))
    su = np.zeros(len(arr), dtype=dtype)
    _numba_dot(arr, dimshape, len(arr), len(dimshape), su)
    return su

M = aa.max(axis=0)+1
idx = np.isin(numba_dimreduce(aa, M), numba_dimreduce(bb, M))
def searchsort(A, B):
    sidx = A.argsort()
    out = np.zeros(len(A), dtype=bool)
    s = np.searchsorted(A, B, sorter=sidx)    
    out[sidx[s]] = True
    return out
def masking(A, B):
    mask = np.empty(shape=np.prod(M), dtype=bool)
    mask[A] = 0
    mask[B] = 1
    return mask[A]
def isin2D(a, b, view=_view1D, search=np.isin):
    # choose your optional view and search
    A, B = view(a), view(b)
    idx = search(A, B)
    return idx
def isin2D(a, b):
    M = a.max(axis=0) + 1
    A = np.ravel_multi_index(a.T, M, order='C')
    B = np.ravel_multi_index(b.T, M, order='C')
    idx = np.isin(A, B)
    return idx

idx = isin2D(aa, bb)
aa[~idx]
>>> 
array([[ 3,  4],
       [48, 59]])
import numpy as np
from numba import njit, prange
import benchit
%matplotlib inline
benchit.setparams(rep=3)

sizes = [100000, 300000, 500000, 1000000, 1700000, 3000000, 5000000]#, 10000000, 30000000]
N = sizes[-1]
aa = np.random.randint(0, 1000000, size=(N, 2))
r = np.random.randint(0, 2, size=N).astype(bool)
M = aa.max(axis=0) + 1

def _view1D(arr, dtype=np.uint32):
    a = np.ascontiguousarray(arr, dtype=dtype)
    dt = np.dtype([('', dtype)]*2)
    return a.view(dt).ravel()

def _numpy_dimreduce(arr, M):
    return np.ravel_multi_index(arr.T, M, order='C')

@njit(parallel=True)
def _numba_dot(arr, dimshape, len_arr, len_dimshape, su):
    for i in prange(len_arr):
        for j in range(len_dimshape):
            su[i] = su[i] + arr[i][j] * dimshape[j]
        
def _numba_dimreduce(arr, M, dtype=np.int64):
    '''not safe if allocation is exceeded'''
    dimshape = np.cumprod(np.insert(M[:-1], 0, 1))
    su = np.zeros(len(arr), dtype=dtype)
    _numba_dot(arr, dimshape, len(arr), len(dimshape), su)
    return su


def isin2D(a, b, view=_view1D, search=np.isin):
    A, B = view(a), view(b)
    idx = search(A, B)
    return idx

def searchsort(A, B):
    sidx = A.argsort()
    out = np.zeros(len(A), dtype=bool)
    s = np.searchsorted(A, B, sorter=sidx)    
    out[sidx[s]] = True
    return out

def isin2D_numpy_view(a, b): return isin2D(a, b, view=_view1D, search=np.isin)

def isin2D_numpy_dimr_isin(a, b, M=M): return isin2D(a, b, view=lambda arr, M=M: _numpy_dimreduce(arr, M), search=np.isin)
def isin2D_numba_dimr_isin(a, b, M=M): return isin2D(a, b, view=lambda arr, M=M: _numba_dimreduce(arr, M), search=np.isin)

def isin2D_numpy_dimr_searchsort(a, b, M=M): return isin2D(a, b, view=lambda arr, M=M: _numpy_dimreduce(arr, M), search=searchsort)
def isin2D_numba_dimr_searchsort(a, b, M=M): return isin2D(a, b, view=lambda arr, M=M: _numba_dimreduce(arr[:, ::-1], M[::-1]), search=searchsort)

def isin2D_numpy_dimr_in1D(a, b, M=M): return isin2D(a, b, view=lambda arr, M=M: _numpy_dimreduce(arr, M), search=np.in1d)
def isin2D_numba_dimr_in1D(a, b, M=M): return isin2D(a, b, view=lambda arr, M=M: _numba_dimreduce(arr[:, ::-1], M[::-1]), search=np.in1d)

fns = [isin2D_numpy_dimr, isin2D_numba_dimr, isin2D_numpy_view, isin2D_numpy_dimr_searchsort, isin2D_numba_dimr_searchsort, isin2D_numpy_dimr_in1D, isin2D_numba_dimr_in1D]

in_ = {s/1000000: (aa[:s], aa[:s][r[:s]]) for s in sizes}
t = benchit.timings(fns, in_, multivar=True, input_name='Millions of events')
t.plot(logx=True, figsize=(12, 6), fontsize=14)
def masking1D(a, b):
    M = aa.max(axis=0) + 1
    A = np.ravel_multi_index(a.T, M, order='C')
    B = np.ravel_multi_index(b.T, M, order='C')
    mask = np.empty(shape=np.prod(M), dtype=bool)
    mask[A] = 1
    mask[B] = 0
    return mask[A]

def masking2D(a, b):
    M = a.max(axis=0) + 1
    mask = np.empty(shape=M, dtype=bool)
    mask[tuple(a.T)] = 1
    mask[tuple(b.T)] = 0
    return mask[tuple(a.T)]

aa = np.array([[988, 986], [299, 327], [756, 30], [327, 189], [988, 294], [64, 232], [11, 46], [38, 223], [933, 770], [141, 141]])
bb = aa[[1, 2, 3, 6, 7, 4]]
masking1D(aa, bb)
>>> array([ True, False, False, False, False,  True, False, False,  True, True])
masking2D(aa, bb)
>>> array([ True, False, False, False, False,  True, False, False,  True, True])
-----------------------
aa = np.array([[1,2],[3,4],[9,10],[48,59]])
bb = np.array([[1,2],[9,10]])
idx
>>> array([ True, False,  True, False])
aa[~idx]
>>> 
array([[ 3,  4],
       [48, 59]])
def _view1D(a, b): # a, b are arrays
    a = np.ascontiguousarray(a, dtype=np.uint32)
    b = np.ascontiguousarray(b, dtype=np.uint32)
    uint32_dt = np.dtype([('', np.uint32)]*2)
    return a.view(uint32_dt).ravel(),  b.view(uint32_dt).ravel()

A, B = _view1D(aa, bb)
idx = np.isin(A, B)
def numpy_dimreduce(arr, M):
    return np.ravel_multi_index(arr.T, M, order='F')

M = aa.max(axis=0)+1
idx = np.isin(numpy_dimreduce(aa, M), numpy_dimreduce(bb, M))
@njit(parallel=True)
def _numba_dot(arr, dimshape, len_arr, len_dimshape, su):
    for i in prange(len_arr):
        for j in range(len_dimshape):
            su[i] = su[i] + arr[i][j] * dimshape[j]
        
def numba_dimreduce(arr, M, dtype=np.int64):
    '''not safe if allocation is exceeded'''
    dimshape = np.cumprod(np.insert(M[:-1], 0, 1))
    su = np.zeros(len(arr), dtype=dtype)
    _numba_dot(arr, dimshape, len(arr), len(dimshape), su)
    return su

M = aa.max(axis=0)+1
idx = np.isin(numba_dimreduce(aa, M), numba_dimreduce(bb, M))
def searchsort(A, B):
    sidx = A.argsort()
    out = np.zeros(len(A), dtype=bool)
    s = np.searchsorted(A, B, sorter=sidx)    
    out[sidx[s]] = True
    return out
def masking(A, B):
    mask = np.empty(shape=np.prod(M), dtype=bool)
    mask[A] = 0
    mask[B] = 1
    return mask[A]
def isin2D(a, b, view=_view1D, search=np.isin):
    # choose your optional view and search
    A, B = view(a), view(b)
    idx = search(A, B)
    return idx
def isin2D(a, b):
    M = a.max(axis=0) + 1
    A = np.ravel_multi_index(a.T, M, order='C')
    B = np.ravel_multi_index(b.T, M, order='C')
    idx = np.isin(A, B)
    return idx

idx = isin2D(aa, bb)
aa[~idx]
>>> 
array([[ 3,  4],
       [48, 59]])
import numpy as np
from numba import njit, prange
import benchit
%matplotlib inline
benchit.setparams(rep=3)

sizes = [100000, 300000, 500000, 1000000, 1700000, 3000000, 5000000]#, 10000000, 30000000]
N = sizes[-1]
aa = np.random.randint(0, 1000000, size=(N, 2))
r = np.random.randint(0, 2, size=N).astype(bool)
M = aa.max(axis=0) + 1

def _view1D(arr, dtype=np.uint32):
    a = np.ascontiguousarray(arr, dtype=dtype)
    dt = np.dtype([('', dtype)]*2)
    return a.view(dt).ravel()

def _numpy_dimreduce(arr, M):
    return np.ravel_multi_index(arr.T, M, order='C')

@njit(parallel=True)
def _numba_dot(arr, dimshape, len_arr, len_dimshape, su):
    for i in prange(len_arr):
        for j in range(len_dimshape):
            su[i] = su[i] + arr[i][j] * dimshape[j]
        
def _numba_dimreduce(arr, M, dtype=np.int64):
    '''not safe if allocation is exceeded'''
    dimshape = np.cumprod(np.insert(M[:-1], 0, 1))
    su = np.zeros(len(arr), dtype=dtype)
    _numba_dot(arr, dimshape, len(arr), len(dimshape), su)
    return su


def isin2D(a, b, view=_view1D, search=np.isin):
    A, B = view(a), view(b)
    idx = search(A, B)
    return idx

def searchsort(A, B):
    sidx = A.argsort()
    out = np.zeros(len(A), dtype=bool)
    s = np.searchsorted(A, B, sorter=sidx)    
    out[sidx[s]] = True
    return out

def isin2D_numpy_view(a, b): return isin2D(a, b, view=_view1D, search=np.isin)

def isin2D_numpy_dimr_isin(a, b, M=M): return isin2D(a, b, view=lambda arr, M=M: _numpy_dimreduce(arr, M), search=np.isin)
def isin2D_numba_dimr_isin(a, b, M=M): return isin2D(a, b, view=lambda arr, M=M: _numba_dimreduce(arr, M), search=np.isin)

def isin2D_numpy_dimr_searchsort(a, b, M=M): return isin2D(a, b, view=lambda arr, M=M: _numpy_dimreduce(arr, M), search=searchsort)
def isin2D_numba_dimr_searchsort(a, b, M=M): return isin2D(a, b, view=lambda arr, M=M: _numba_dimreduce(arr[:, ::-1], M[::-1]), search=searchsort)

def isin2D_numpy_dimr_in1D(a, b, M=M): return isin2D(a, b, view=lambda arr, M=M: _numpy_dimreduce(arr, M), search=np.in1d)
def isin2D_numba_dimr_in1D(a, b, M=M): return isin2D(a, b, view=lambda arr, M=M: _numba_dimreduce(arr[:, ::-1], M[::-1]), search=np.in1d)

fns = [isin2D_numpy_dimr, isin2D_numba_dimr, isin2D_numpy_view, isin2D_numpy_dimr_searchsort, isin2D_numba_dimr_searchsort, isin2D_numpy_dimr_in1D, isin2D_numba_dimr_in1D]

in_ = {s/1000000: (aa[:s], aa[:s][r[:s]]) for s in sizes}
t = benchit.timings(fns, in_, multivar=True, input_name='Millions of events')
t.plot(logx=True, figsize=(12, 6), fontsize=14)
def masking1D(a, b):
    M = aa.max(axis=0) + 1
    A = np.ravel_multi_index(a.T, M, order='C')
    B = np.ravel_multi_index(b.T, M, order='C')
    mask = np.empty(shape=np.prod(M), dtype=bool)
    mask[A] = 1
    mask[B] = 0
    return mask[A]

def masking2D(a, b):
    M = a.max(axis=0) + 1
    mask = np.empty(shape=M, dtype=bool)
    mask[tuple(a.T)] = 1
    mask[tuple(b.T)] = 0
    return mask[tuple(a.T)]

aa = np.array([[988, 986], [299, 327], [756, 30], [327, 189], [988, 294], [64, 232], [11, 46], [38, 223], [933, 770], [141, 141]])
bb = aa[[1, 2, 3, 6, 7, 4]]
masking1D(aa, bb)
>>> array([ True, False, False, False, False,  True, False, False,  True, True])
masking2D(aa, bb)
>>> array([ True, False, False, False, False,  True, False, False,  True, True])
-----------------------
aa = np.array([[1,2],[3,4],[9,10],[48,59]])
bb = np.array([[1,2],[9,10]])
idx
>>> array([ True, False,  True, False])
aa[~idx]
>>> 
array([[ 3,  4],
       [48, 59]])
def _view1D(a, b): # a, b are arrays
    a = np.ascontiguousarray(a, dtype=np.uint32)
    b = np.ascontiguousarray(b, dtype=np.uint32)
    uint32_dt = np.dtype([('', np.uint32)]*2)
    return a.view(uint32_dt).ravel(),  b.view(uint32_dt).ravel()

A, B = _view1D(aa, bb)
idx = np.isin(A, B)
def numpy_dimreduce(arr, M):
    return np.ravel_multi_index(arr.T, M, order='F')

M = aa.max(axis=0)+1
idx = np.isin(numpy_dimreduce(aa, M), numpy_dimreduce(bb, M))
@njit(parallel=True)
def _numba_dot(arr, dimshape, len_arr, len_dimshape, su):
    for i in prange(len_arr):
        for j in range(len_dimshape):
            su[i] = su[i] + arr[i][j] * dimshape[j]
        
def numba_dimreduce(arr, M, dtype=np.int64):
    '''not safe if allocation is exceeded'''
    dimshape = np.cumprod(np.insert(M[:-1], 0, 1))
    su = np.zeros(len(arr), dtype=dtype)
    _numba_dot(arr, dimshape, len(arr), len(dimshape), su)
    return su

M = aa.max(axis=0)+1
idx = np.isin(numba_dimreduce(aa, M), numba_dimreduce(bb, M))
def searchsort(A, B):
    sidx = A.argsort()
    out = np.zeros(len(A), dtype=bool)
    s = np.searchsorted(A, B, sorter=sidx)    
    out[sidx[s]] = True
    return out
def masking(A, B):
    mask = np.empty(shape=np.prod(M), dtype=bool)
    mask[A] = 0
    mask[B] = 1
    return mask[A]
def isin2D(a, b, view=_view1D, search=np.isin):
    # choose your optional view and search
    A, B = view(a), view(b)
    idx = search(A, B)
    return idx
def isin2D(a, b):
    M = a.max(axis=0) + 1
    A = np.ravel_multi_index(a.T, M, order='C')
    B = np.ravel_multi_index(b.T, M, order='C')
    idx = np.isin(A, B)
    return idx

idx = isin2D(aa, bb)
aa[~idx]
>>> 
array([[ 3,  4],
       [48, 59]])
import numpy as np
from numba import njit, prange
import benchit
%matplotlib inline
benchit.setparams(rep=3)

sizes = [100000, 300000, 500000, 1000000, 1700000, 3000000, 5000000]#, 10000000, 30000000]
N = sizes[-1]
aa = np.random.randint(0, 1000000, size=(N, 2))
r = np.random.randint(0, 2, size=N).astype(bool)
M = aa.max(axis=0) + 1

def _view1D(arr, dtype=np.uint32):
    a = np.ascontiguousarray(arr, dtype=dtype)
    dt = np.dtype([('', dtype)]*2)
    return a.view(dt).ravel()

def _numpy_dimreduce(arr, M):
    return np.ravel_multi_index(arr.T, M, order='C')

@njit(parallel=True)
def _numba_dot(arr, dimshape, len_arr, len_dimshape, su):
    for i in prange(len_arr):
        for j in range(len_dimshape):
            su[i] = su[i] + arr[i][j] * dimshape[j]
        
def _numba_dimreduce(arr, M, dtype=np.int64):
    '''not safe if allocation is exceeded'''
    dimshape = np.cumprod(np.insert(M[:-1], 0, 1))
    su = np.zeros(len(arr), dtype=dtype)
    _numba_dot(arr, dimshape, len(arr), len(dimshape), su)
    return su


def isin2D(a, b, view=_view1D, search=np.isin):
    A, B = view(a), view(b)
    idx = search(A, B)
    return idx

def searchsort(A, B):
    sidx = A.argsort()
    out = np.zeros(len(A), dtype=bool)
    s = np.searchsorted(A, B, sorter=sidx)    
    out[sidx[s]] = True
    return out

def isin2D_numpy_view(a, b): return isin2D(a, b, view=_view1D, search=np.isin)

def isin2D_numpy_dimr_isin(a, b, M=M): return isin2D(a, b, view=lambda arr, M=M: _numpy_dimreduce(arr, M), search=np.isin)
def isin2D_numba_dimr_isin(a, b, M=M): return isin2D(a, b, view=lambda arr, M=M: _numba_dimreduce(arr, M), search=np.isin)

def isin2D_numpy_dimr_searchsort(a, b, M=M): return isin2D(a, b, view=lambda arr, M=M: _numpy_dimreduce(arr, M), search=searchsort)
def isin2D_numba_dimr_searchsort(a, b, M=M): return isin2D(a, b, view=lambda arr, M=M: _numba_dimreduce(arr[:, ::-1], M[::-1]), search=searchsort)

def isin2D_numpy_dimr_in1D(a, b, M=M): return isin2D(a, b, view=lambda arr, M=M: _numpy_dimreduce(arr, M), search=np.in1d)
def isin2D_numba_dimr_in1D(a, b, M=M): return isin2D(a, b, view=lambda arr, M=M: _numba_dimreduce(arr[:, ::-1], M[::-1]), search=np.in1d)

fns = [isin2D_numpy_dimr, isin2D_numba_dimr, isin2D_numpy_view, isin2D_numpy_dimr_searchsort, isin2D_numba_dimr_searchsort, isin2D_numpy_dimr_in1D, isin2D_numba_dimr_in1D]

in_ = {s/1000000: (aa[:s], aa[:s][r[:s]]) for s in sizes}
t = benchit.timings(fns, in_, multivar=True, input_name='Millions of events')
t.plot(logx=True, figsize=(12, 6), fontsize=14)
def masking1D(a, b):
    M = aa.max(axis=0) + 1
    A = np.ravel_multi_index(a.T, M, order='C')
    B = np.ravel_multi_index(b.T, M, order='C')
    mask = np.empty(shape=np.prod(M), dtype=bool)
    mask[A] = 1
    mask[B] = 0
    return mask[A]

def masking2D(a, b):
    M = a.max(axis=0) + 1
    mask = np.empty(shape=M, dtype=bool)
    mask[tuple(a.T)] = 1
    mask[tuple(b.T)] = 0
    return mask[tuple(a.T)]

aa = np.array([[988, 986], [299, 327], [756, 30], [327, 189], [988, 294], [64, 232], [11, 46], [38, 223], [933, 770], [141, 141]])
bb = aa[[1, 2, 3, 6, 7, 4]]
masking1D(aa, bb)
>>> array([ True, False, False, False, False,  True, False, False,  True, True])
masking2D(aa, bb)
>>> array([ True, False, False, False, False,  True, False, False,  True, True])
-----------------------
aa = np.array([[1,2],[3,4],[9,10],[48,59]])
bb = np.array([[1,2],[9,10]])
idx
>>> array([ True, False,  True, False])
aa[~idx]
>>> 
array([[ 3,  4],
       [48, 59]])
def _view1D(a, b): # a, b are arrays
    a = np.ascontiguousarray(a, dtype=np.uint32)
    b = np.ascontiguousarray(b, dtype=np.uint32)
    uint32_dt = np.dtype([('', np.uint32)]*2)
    return a.view(uint32_dt).ravel(),  b.view(uint32_dt).ravel()

A, B = _view1D(aa, bb)
idx = np.isin(A, B)
def numpy_dimreduce(arr, M):
    return np.ravel_multi_index(arr.T, M, order='F')

M = aa.max(axis=0)+1
idx = np.isin(numpy_dimreduce(aa, M), numpy_dimreduce(bb, M))
@njit(parallel=True)
def _numba_dot(arr, dimshape, len_arr, len_dimshape, su):
    for i in prange(len_arr):
        for j in range(len_dimshape):
            su[i] = su[i] + arr[i][j] * dimshape[j]
        
def numba_dimreduce(arr, M, dtype=np.int64):
    '''not safe if allocation is exceeded'''
    dimshape = np.cumprod(np.insert(M[:-1], 0, 1))
    su = np.zeros(len(arr), dtype=dtype)
    _numba_dot(arr, dimshape, len(arr), len(dimshape), su)
    return su

M = aa.max(axis=0)+1
idx = np.isin(numba_dimreduce(aa, M), numba_dimreduce(bb, M))
def searchsort(A, B):
    sidx = A.argsort()
    out = np.zeros(len(A), dtype=bool)
    s = np.searchsorted(A, B, sorter=sidx)    
    out[sidx[s]] = True
    return out
def masking(A, B):
    mask = np.empty(shape=np.prod(M), dtype=bool)
    mask[A] = 0
    mask[B] = 1
    return mask[A]
def isin2D(a, b, view=_view1D, search=np.isin):
    # choose your optional view and search
    A, B = view(a), view(b)
    idx = search(A, B)
    return idx
def isin2D(a, b):
    M = a.max(axis=0) + 1
    A = np.ravel_multi_index(a.T, M, order='C')
    B = np.ravel_multi_index(b.T, M, order='C')
    idx = np.isin(A, B)
    return idx

idx = isin2D(aa, bb)
aa[~idx]
>>> 
array([[ 3,  4],
       [48, 59]])
import numpy as np
from numba import njit, prange
import benchit
%matplotlib inline
benchit.setparams(rep=3)

sizes = [100000, 300000, 500000, 1000000, 1700000, 3000000, 5000000]#, 10000000, 30000000]
N = sizes[-1]
aa = np.random.randint(0, 1000000, size=(N, 2))
r = np.random.randint(0, 2, size=N).astype(bool)
M = aa.max(axis=0) + 1

def _view1D(arr, dtype=np.uint32):
    a = np.ascontiguousarray(arr, dtype=dtype)
    dt = np.dtype([('', dtype)]*2)
    return a.view(dt).ravel()

def _numpy_dimreduce(arr, M):
    return np.ravel_multi_index(arr.T, M, order='C')

@njit(parallel=True)
def _numba_dot(arr, dimshape, len_arr, len_dimshape, su):
    for i in prange(len_arr):
        for j in range(len_dimshape):
            su[i] = su[i] + arr[i][j] * dimshape[j]
        
def _numba_dimreduce(arr, M, dtype=np.int64):
    '''not safe if allocation is exceeded'''
    dimshape = np.cumprod(np.insert(M[:-1], 0, 1))
    su = np.zeros(len(arr), dtype=dtype)
    _numba_dot(arr, dimshape, len(arr), len(dimshape), su)
    return su


def isin2D(a, b, view=_view1D, search=np.isin):
    A, B = view(a), view(b)
    idx = search(A, B)
    return idx

def searchsort(A, B):
    sidx = A.argsort()
    out = np.zeros(len(A), dtype=bool)
    s = np.searchsorted(A, B, sorter=sidx)    
    out[sidx[s]] = True
    return out

def isin2D_numpy_view(a, b): return isin2D(a, b, view=_view1D, search=np.isin)

def isin2D_numpy_dimr_isin(a, b, M=M): return isin2D(a, b, view=lambda arr, M=M: _numpy_dimreduce(arr, M), search=np.isin)
def isin2D_numba_dimr_isin(a, b, M=M): return isin2D(a, b, view=lambda arr, M=M: _numba_dimreduce(arr, M), search=np.isin)

def isin2D_numpy_dimr_searchsort(a, b, M=M): return isin2D(a, b, view=lambda arr, M=M: _numpy_dimreduce(arr, M), search=searchsort)
def isin2D_numba_dimr_searchsort(a, b, M=M): return isin2D(a, b, view=lambda arr, M=M: _numba_dimreduce(arr[:, ::-1], M[::-1]), search=searchsort)

def isin2D_numpy_dimr_in1D(a, b, M=M): return isin2D(a, b, view=lambda arr, M=M: _numpy_dimreduce(arr, M), search=np.in1d)
def isin2D_numba_dimr_in1D(a, b, M=M): return isin2D(a, b, view=lambda arr, M=M: _numba_dimreduce(arr[:, ::-1], M[::-1]), search=np.in1d)

fns = [isin2D_numpy_dimr, isin2D_numba_dimr, isin2D_numpy_view, isin2D_numpy_dimr_searchsort, isin2D_numba_dimr_searchsort, isin2D_numpy_dimr_in1D, isin2D_numba_dimr_in1D]

in_ = {s/1000000: (aa[:s], aa[:s][r[:s]]) for s in sizes}
t = benchit.timings(fns, in_, multivar=True, input_name='Millions of events')
t.plot(logx=True, figsize=(12, 6), fontsize=14)
def masking1D(a, b):
    M = aa.max(axis=0) + 1
    A = np.ravel_multi_index(a.T, M, order='C')
    B = np.ravel_multi_index(b.T, M, order='C')
    mask = np.empty(shape=np.prod(M), dtype=bool)
    mask[A] = 1
    mask[B] = 0
    return mask[A]

def masking2D(a, b):
    M = a.max(axis=0) + 1
    mask = np.empty(shape=M, dtype=bool)
    mask[tuple(a.T)] = 1
    mask[tuple(b.T)] = 0
    return mask[tuple(a.T)]

aa = np.array([[988, 986], [299, 327], [756, 30], [327, 189], [988, 294], [64, 232], [11, 46], [38, 223], [933, 770], [141, 141]])
bb = aa[[1, 2, 3, 6, 7, 4]]
masking1D(aa, bb)
>>> array([ True, False, False, False, False,  True, False, False,  True, True])
masking2D(aa, bb)
>>> array([ True, False, False, False, False,  True, False, False,  True, True])
-----------------------
aa = np.array([[1,2],[3,4],[9,10],[48,59]])
bb = np.array([[1,2],[9,10]])
idx
>>> array([ True, False,  True, False])
aa[~idx]
>>> 
array([[ 3,  4],
       [48, 59]])
def _view1D(a, b): # a, b are arrays
    a = np.ascontiguousarray(a, dtype=np.uint32)
    b = np.ascontiguousarray(b, dtype=np.uint32)
    uint32_dt = np.dtype([('', np.uint32)]*2)
    return a.view(uint32_dt).ravel(),  b.view(uint32_dt).ravel()

A, B = _view1D(aa, bb)
idx = np.isin(A, B)
def numpy_dimreduce(arr, M):
    return np.ravel_multi_index(arr.T, M, order='F')

M = aa.max(axis=0)+1
idx = np.isin(numpy_dimreduce(aa, M), numpy_dimreduce(bb, M))
@njit(parallel=True)
def _numba_dot(arr, dimshape, len_arr, len_dimshape, su):
    for i in prange(len_arr):
        for j in range(len_dimshape):
            su[i] = su[i] + arr[i][j] * dimshape[j]
        
def numba_dimreduce(arr, M, dtype=np.int64):
    '''not safe if allocation is exceeded'''
    dimshape = np.cumprod(np.insert(M[:-1], 0, 1))
    su = np.zeros(len(arr), dtype=dtype)
    _numba_dot(arr, dimshape, len(arr), len(dimshape), su)
    return su

M = aa.max(axis=0)+1
idx = np.isin(numba_dimreduce(aa, M), numba_dimreduce(bb, M))
def searchsort(A, B):
    sidx = A.argsort()
    out = np.zeros(len(A), dtype=bool)
    s = np.searchsorted(A, B, sorter=sidx)    
    out[sidx[s]] = True
    return out
def masking(A, B):
    mask = np.empty(shape=np.prod(M), dtype=bool)
    mask[A] = 0
    mask[B] = 1
    return mask[A]
def isin2D(a, b, view=_view1D, search=np.isin):
    # choose your optional view and search
    A, B = view(a), view(b)
    idx = search(A, B)
    return idx
def isin2D(a, b):
    M = a.max(axis=0) + 1
    A = np.ravel_multi_index(a.T, M, order='C')
    B = np.ravel_multi_index(b.T, M, order='C')
    idx = np.isin(A, B)
    return idx

idx = isin2D(aa, bb)
aa[~idx]
>>> 
array([[ 3,  4],
       [48, 59]])
import numpy as np
from numba import njit, prange
import benchit
%matplotlib inline
benchit.setparams(rep=3)

sizes = [100000, 300000, 500000, 1000000, 1700000, 3000000, 5000000]#, 10000000, 30000000]
N = sizes[-1]
aa = np.random.randint(0, 1000000, size=(N, 2))
r = np.random.randint(0, 2, size=N).astype(bool)
M = aa.max(axis=0) + 1

def _view1D(arr, dtype=np.uint32):
    a = np.ascontiguousarray(arr, dtype=dtype)
    dt = np.dtype([('', dtype)]*2)
    return a.view(dt).ravel()

def _numpy_dimreduce(arr, M):
    return np.ravel_multi_index(arr.T, M, order='C')

@njit(parallel=True)
def _numba_dot(arr, dimshape, len_arr, len_dimshape, su):
    for i in prange(len_arr):
        for j in range(len_dimshape):
            su[i] = su[i] + arr[i][j] * dimshape[j]
        
def _numba_dimreduce(arr, M, dtype=np.int64):
    '''not safe if allocation is exceeded'''
    dimshape = np.cumprod(np.insert(M[:-1], 0, 1))
    su = np.zeros(len(arr), dtype=dtype)
    _numba_dot(arr, dimshape, len(arr), len(dimshape), su)
    return su


def isin2D(a, b, view=_view1D, search=np.isin):
    A, B = view(a), view(b)
    idx = search(A, B)
    return idx

def searchsort(A, B):
    sidx = A.argsort()
    out = np.zeros(len(A), dtype=bool)
    s = np.searchsorted(A, B, sorter=sidx)    
    out[sidx[s]] = True
    return out

def isin2D_numpy_view(a, b): return isin2D(a, b, view=_view1D, search=np.isin)

def isin2D_numpy_dimr_isin(a, b, M=M): return isin2D(a, b, view=lambda arr, M=M: _numpy_dimreduce(arr, M), search=np.isin)
def isin2D_numba_dimr_isin(a, b, M=M): return isin2D(a, b, view=lambda arr, M=M: _numba_dimreduce(arr, M), search=np.isin)

def isin2D_numpy_dimr_searchsort(a, b, M=M): return isin2D(a, b, view=lambda arr, M=M: _numpy_dimreduce(arr, M), search=searchsort)
def isin2D_numba_dimr_searchsort(a, b, M=M): return isin2D(a, b, view=lambda arr, M=M: _numba_dimreduce(arr[:, ::-1], M[::-1]), search=searchsort)

def isin2D_numpy_dimr_in1D(a, b, M=M): return isin2D(a, b, view=lambda arr, M=M: _numpy_dimreduce(arr, M), search=np.in1d)
def isin2D_numba_dimr_in1D(a, b, M=M): return isin2D(a, b, view=lambda arr, M=M: _numba_dimreduce(arr[:, ::-1], M[::-1]), search=np.in1d)

fns = [isin2D_numpy_dimr, isin2D_numba_dimr, isin2D_numpy_view, isin2D_numpy_dimr_searchsort, isin2D_numba_dimr_searchsort, isin2D_numpy_dimr_in1D, isin2D_numba_dimr_in1D]

in_ = {s/1000000: (aa[:s], aa[:s][r[:s]]) for s in sizes}
t = benchit.timings(fns, in_, multivar=True, input_name='Millions of events')
t.plot(logx=True, figsize=(12, 6), fontsize=14)
def masking1D(a, b):
    M = aa.max(axis=0) + 1
    A = np.ravel_multi_index(a.T, M, order='C')
    B = np.ravel_multi_index(b.T, M, order='C')
    mask = np.empty(shape=np.prod(M), dtype=bool)
    mask[A] = 1
    mask[B] = 0
    return mask[A]

def masking2D(a, b):
    M = a.max(axis=0) + 1
    mask = np.empty(shape=M, dtype=bool)
    mask[tuple(a.T)] = 1
    mask[tuple(b.T)] = 0
    return mask[tuple(a.T)]

aa = np.array([[988, 986], [299, 327], [756, 30], [327, 189], [988, 294], [64, 232], [11, 46], [38, 223], [933, 770], [141, 141]])
bb = aa[[1, 2, 3, 6, 7, 4]]
masking1D(aa, bb)
>>> array([ True, False, False, False, False,  True, False, False,  True, True])
masking2D(aa, bb)
>>> array([ True, False, False, False, False,  True, False, False,  True, True])
-----------------------
aa = np.array([[1,2],[3,4],[9,10],[48,59]])
bb = np.array([[1,2],[9,10]])
idx
>>> array([ True, False,  True, False])
aa[~idx]
>>> 
array([[ 3,  4],
       [48, 59]])
def _view1D(a, b): # a, b are arrays
    a = np.ascontiguousarray(a, dtype=np.uint32)
    b = np.ascontiguousarray(b, dtype=np.uint32)
    uint32_dt = np.dtype([('', np.uint32)]*2)
    return a.view(uint32_dt).ravel(),  b.view(uint32_dt).ravel()

A, B = _view1D(aa, bb)
idx = np.isin(A, B)
def numpy_dimreduce(arr, M):
    return np.ravel_multi_index(arr.T, M, order='F')

M = aa.max(axis=0)+1
idx = np.isin(numpy_dimreduce(aa, M), numpy_dimreduce(bb, M))
@njit(parallel=True)
def _numba_dot(arr, dimshape, len_arr, len_dimshape, su):
    for i in prange(len_arr):
        for j in range(len_dimshape):
            su[i] = su[i] + arr[i][j] * dimshape[j]
        
def numba_dimreduce(arr, M, dtype=np.int64):
    '''not safe if allocation is exceeded'''
    dimshape = np.cumprod(np.insert(M[:-1], 0, 1))
    su = np.zeros(len(arr), dtype=dtype)
    _numba_dot(arr, dimshape, len(arr), len(dimshape), su)
    return su

M = aa.max(axis=0)+1
idx = np.isin(numba_dimreduce(aa, M), numba_dimreduce(bb, M))
def searchsort(A, B):
    sidx = A.argsort()
    out = np.zeros(len(A), dtype=bool)
    s = np.searchsorted(A, B, sorter=sidx)    
    out[sidx[s]] = True
    return out
def masking(A, B):
    mask = np.empty(shape=np.prod(M), dtype=bool)
    mask[A] = 0
    mask[B] = 1
    return mask[A]
def isin2D(a, b, view=_view1D, search=np.isin):
    # choose your optional view and search
    A, B = view(a), view(b)
    idx = search(A, B)
    return idx
def isin2D(a, b):
    M = a.max(axis=0) + 1
    A = np.ravel_multi_index(a.T, M, order='C')
    B = np.ravel_multi_index(b.T, M, order='C')
    idx = np.isin(A, B)
    return idx

idx = isin2D(aa, bb)
aa[~idx]
>>> 
array([[ 3,  4],
       [48, 59]])
import numpy as np
from numba import njit, prange
import benchit
%matplotlib inline
benchit.setparams(rep=3)

sizes = [100000, 300000, 500000, 1000000, 1700000, 3000000, 5000000]#, 10000000, 30000000]
N = sizes[-1]
aa = np.random.randint(0, 1000000, size=(N, 2))
r = np.random.randint(0, 2, size=N).astype(bool)
M = aa.max(axis=0) + 1

def _view1D(arr, dtype=np.uint32):
    a = np.ascontiguousarray(arr, dtype=dtype)
    dt = np.dtype([('', dtype)]*2)
    return a.view(dt).ravel()

def _numpy_dimreduce(arr, M):
    return np.ravel_multi_index(arr.T, M, order='C')

@njit(parallel=True)
def _numba_dot(arr, dimshape, len_arr, len_dimshape, su):
    for i in prange(len_arr):
        for j in range(len_dimshape):
            su[i] = su[i] + arr[i][j] * dimshape[j]
        
def _numba_dimreduce(arr, M, dtype=np.int64):
    '''not safe if allocation is exceeded'''
    dimshape = np.cumprod(np.insert(M[:-1], 0, 1))
    su = np.zeros(len(arr), dtype=dtype)
    _numba_dot(arr, dimshape, len(arr), len(dimshape), su)
    return su


def isin2D(a, b, view=_view1D, search=np.isin):
    A, B = view(a), view(b)
    idx = search(A, B)
    return idx

def searchsort(A, B):
    sidx = A.argsort()
    out = np.zeros(len(A), dtype=bool)
    s = np.searchsorted(A, B, sorter=sidx)    
    out[sidx[s]] = True
    return out

def isin2D_numpy_view(a, b): return isin2D(a, b, view=_view1D, search=np.isin)

def isin2D_numpy_dimr_isin(a, b, M=M): return isin2D(a, b, view=lambda arr, M=M: _numpy_dimreduce(arr, M), search=np.isin)
def isin2D_numba_dimr_isin(a, b, M=M): return isin2D(a, b, view=lambda arr, M=M: _numba_dimreduce(arr, M), search=np.isin)

def isin2D_numpy_dimr_searchsort(a, b, M=M): return isin2D(a, b, view=lambda arr, M=M: _numpy_dimreduce(arr, M), search=searchsort)
def isin2D_numba_dimr_searchsort(a, b, M=M): return isin2D(a, b, view=lambda arr, M=M: _numba_dimreduce(arr[:, ::-1], M[::-1]), search=searchsort)

def isin2D_numpy_dimr_in1D(a, b, M=M): return isin2D(a, b, view=lambda arr, M=M: _numpy_dimreduce(arr, M), search=np.in1d)
def isin2D_numba_dimr_in1D(a, b, M=M): return isin2D(a, b, view=lambda arr, M=M: _numba_dimreduce(arr[:, ::-1], M[::-1]), search=np.in1d)

fns = [isin2D_numpy_dimr, isin2D_numba_dimr, isin2D_numpy_view, isin2D_numpy_dimr_searchsort, isin2D_numba_dimr_searchsort, isin2D_numpy_dimr_in1D, isin2D_numba_dimr_in1D]

in_ = {s/1000000: (aa[:s], aa[:s][r[:s]]) for s in sizes}
t = benchit.timings(fns, in_, multivar=True, input_name='Millions of events')
t.plot(logx=True, figsize=(12, 6), fontsize=14)
def masking1D(a, b):
    M = aa.max(axis=0) + 1
    A = np.ravel_multi_index(a.T, M, order='C')
    B = np.ravel_multi_index(b.T, M, order='C')
    mask = np.empty(shape=np.prod(M), dtype=bool)
    mask[A] = 1
    mask[B] = 0
    return mask[A]

def masking2D(a, b):
    M = a.max(axis=0) + 1
    mask = np.empty(shape=M, dtype=bool)
    mask[tuple(a.T)] = 1
    mask[tuple(b.T)] = 0
    return mask[tuple(a.T)]

aa = np.array([[988, 986], [299, 327], [756, 30], [327, 189], [988, 294], [64, 232], [11, 46], [38, 223], [933, 770], [141, 141]])
bb = aa[[1, 2, 3, 6, 7, 4]]
masking1D(aa, bb)
>>> array([ True, False, False, False, False,  True, False, False,  True, True])
masking2D(aa, bb)
>>> array([ True, False, False, False, False,  True, False, False,  True, True])

Whether to clone an object for a prototype

copy iconCopydownload iconDownload
      +------------------+      +-----animal-----+      +-------------------+
a --->|  [[Prototype]] -------->|  [[Prototype]]------->|  Object.prototype |
      |  name: 'Annie'   |      |  breathe() {}  |      |                   |
      +------------------+      +----------------+      +-------------------+
const animal = {
  breathe() {console.log('The animal is breathing.')},
}
// construtor
function Dog(name) {
  this.name = name;
}

Dog.prototype = animal; // without Object.create()
Dog.prototype.bark = () => console.log("Woof!"); // this updates  `animal` also (not good!)

let a = new Dog('Annie');
a.breathe();
a.bark();
      +------------------+      +----------------+      +------animal-------+
a --->|  [[Prototype]] -------->|  [[Prototype]]------->|   [[Prototype]]---------+
      |  name: 'Annie'   |      |  <empty>       |      |    breathe() {}   |     |
      +------------------+      +----------------+      +-------------------+     |
                                                                                  |
      +---------------------------------------------------------------------------+
      v
      +------------------+
      | Object.prototype |
      |                  |
      +------------------+
-----------------------
      +------------------+      +-----animal-----+      +-------------------+
a --->|  [[Prototype]] -------->|  [[Prototype]]------->|  Object.prototype |
      |  name: 'Annie'   |      |  breathe() {}  |      |                   |
      +------------------+      +----------------+      +-------------------+
const animal = {
  breathe() {console.log('The animal is breathing.')},
}
// construtor
function Dog(name) {
  this.name = name;
}

Dog.prototype = animal; // without Object.create()
Dog.prototype.bark = () => console.log("Woof!"); // this updates  `animal` also (not good!)

let a = new Dog('Annie');
a.breathe();
a.bark();
      +------------------+      +----------------+      +------animal-------+
a --->|  [[Prototype]] -------->|  [[Prototype]]------->|   [[Prototype]]---------+
      |  name: 'Annie'   |      |  <empty>       |      |    breathe() {}   |     |
      +------------------+      +----------------+      +-------------------+     |
                                                                                  |
      +---------------------------------------------------------------------------+
      v
      +------------------+
      | Object.prototype |
      |                  |
      +------------------+
-----------------------
      +------------------+      +-----animal-----+      +-------------------+
a --->|  [[Prototype]] -------->|  [[Prototype]]------->|  Object.prototype |
      |  name: 'Annie'   |      |  breathe() {}  |      |                   |
      +------------------+      +----------------+      +-------------------+
const animal = {
  breathe() {console.log('The animal is breathing.')},
}
// construtor
function Dog(name) {
  this.name = name;
}

Dog.prototype = animal; // without Object.create()
Dog.prototype.bark = () => console.log("Woof!"); // this updates  `animal` also (not good!)

let a = new Dog('Annie');
a.breathe();
a.bark();
      +------------------+      +----------------+      +------animal-------+
a --->|  [[Prototype]] -------->|  [[Prototype]]------->|   [[Prototype]]---------+
      |  name: 'Annie'   |      |  <empty>       |      |    breathe() {}   |     |
      +------------------+      +----------------+      +-------------------+     |
                                                                                  |
      +---------------------------------------------------------------------------+
      v
      +------------------+
      | Object.prototype |
      |                  |
      +------------------+

Setting up multi-release JAR unit tests

copy iconCopydownload iconDownload
import static org.junit.jupiter.api.Assertions.assertFalse;
import static org.junit.jupiter.api.Assertions.assertTrue;
import static org.junit.jupiter.api.Assumptions.assumeTrue;

import org.junit.jupiter.api.Test;

class RecordHelperTest
{

    @Test
    void isNotARecord()
    {
        assertFalse( RecordHelper.isRecord(Object.class));
    }

    @Test
    void isARecord() throws Exception
    {
        assumeTrue( Integer.parseInt( System.getProperty( "java.specification.version" ) ) >= 16 );
        
        Class c = Class.forName( "jdk.net.UnixDomainPrincipal" );
        assertTrue( RecordHelper.isRecord(c));
    }

}
-----------------------
        <profile>
            <id>modules-jdk8</id>
            <activation>
                <jdk>[1.8,11)</jdk>
            </activation>
            <modules>
                <module>core</module>
            </modules>
        </profile>

        <profile>
            <id>modules-jdk16</id>
            <activation>
                <jdk>[16,17)</jdk>
            </activation>
            <modules>
                <module>core</module>
                <module>m16</module>
            </modules>
        </profile>

        <profile>
            <id>modules-jdk17</id>
            <activation>
                <jdk>[17,)</jdk>
            </activation>
            <modules>
                <module>core</module>
                <module>m16</module>
                <module>m17</module>
                <module>release</module>
            </modules>
        </profile>

Creating a module subclass in a Python extension

copy iconCopydownload iconDownload
...
// copied from moduleobject.c
static int
module_init_dict(struct testmod_s* mod, PyObject* md_dict,
    PyObject* name, PyObject* doc)
{
    _Py_IDENTIFIER(__name__);
    _Py_IDENTIFIER(__doc__);
    _Py_IDENTIFIER(__package__);
    _Py_IDENTIFIER(__loader__);
    _Py_IDENTIFIER(__spec__);

    if (md_dict == NULL)
        return -1;
    if (doc == NULL)
        doc = Py_None;

    if (_PyDict_SetItemId(md_dict, &PyId___name__, name) != 0)
        return -1;
    if (_PyDict_SetItemId(md_dict, &PyId___doc__, doc) != 0)
        return -1;
    if (_PyDict_SetItemId(md_dict, &PyId___package__, Py_None) != 0)
        return -1;
    if (_PyDict_SetItemId(md_dict, &PyId___loader__, Py_None) != 0)
        return -1;
    if (_PyDict_SetItemId(md_dict, &PyId___spec__, Py_None) != 0)
        return -1;
    if (PyUnicode_CheckExact(name)) {
        Py_INCREF(name);
        Py_XSETREF(mod->md_name, name);
    }

    return 0;
}

static PyObject* testmod_create(PyObject* spec, PyModuleDef* def)
{
    struct testmod_s* module = PyObject_GC_New(struct testmod_s, &testmodtype);
    if (module == NULL) {
        return NULL;
    }
    PyObject* name = PyUnicode_FromString("testmod");
    if (name == NULL) {
        Py_DECREF(module);
        return 0;
    }
    module->md_def = NULL;
    module->md_state = NULL;
    module->md_weaklist = NULL;
    module->md_name = NULL;
    module->md_dict = PyDict_New();
    int cr = module_init_dict(module, module->md_dict, name, NULL);
    Py_DECREF(name);
    if (cr != 0) {
        Py_DECREF(module);
        return NULL;
    }

    return (PyObject*)module;
}

Angular: How to route different modules on the same path depending on service

copy iconCopydownload iconDownload
import { NgModule } from '@angular/core';
import { RouterModule } from '@angular/router';
import { BrowserModule } from '@angular/platform-browser';

import { AppComponent } from './app.component';
import { AorBActivateGuard } from './aorb-activate.guard';
import { ABService } from './ab.service';

@NgModule({
  imports: [
    BrowserModule,
    RouterModule.forRoot([
      {
        path: 'aorb',
        loadChildren: () => {
          if ((new ABService()).isAResponsible()){
            return import('./a/a.module').then((m) => m.AModule)
          } else {
            return import('./b/b.module').then((m) => m.BModule)
          }
        },
      },
    ]),
  ],
  providers: [AorBActivateGuard, ABService],
  declarations: [AppComponent],
  bootstrap: [AppComponent],
})
export class AppModule {}
-----------------------
[...]
@NgModule({
  imports: [
    BrowserModule,
    RouterModule.forRoot([], {
      initialNavigation: 'disabled',
    }),
  ],
  providers: [ABService, BooleanStorageService],
  declarations: [AppComponent],
  bootstrap: [AppComponent],
})
export class AppModule {
  constructor(private router: Router, private abService: ABService) {
    abService.onABChange.subscribe(this.resetRouterConfig(true).bind(this));
    this.resetRouterConfig(false)();
    this.router.initialNavigation();
  }

  reloadCurrentRoute() {
    const currentUrl = this.router.url;
    this.router.navigateByUrl('/', { skipLocationChange: true }).then(() => {
      this.router.navigate([currentUrl]);
    });
  }

  resetRouterConfig(refresh: boolean) {
    return () => {
      const constructedConfig = [
        {
          path: '',
          pathMatch: 'full',
          redirectTo: 'aorb',
        },
        {
          path: 'aorb',
          loadChildren: () => {
            if (this.abService.isAResponsible()) {
              return import('./a/a.module').then((m) => m.AModule);
            } else {
              return import('./b/b.module').then((m) => m.BModule);
            }
          },
        },
      ];

      this.router.resetConfig(constructedConfig);

      if (refresh) this.reloadCurrentRoute();
    };
  }
}

Community Discussions

Trending Discussions on preferable
  • Pandas DataFrame update cell values from second DataFrame
  • How to return const&amp; from std::visit?
  • How can I get a list of all strings &quot;greater&quot; than a specific value in a List&lt;string&gt;?
  • MongoDB and LINQ: &quot;NOT IN&quot; clause
  • How can I extend the quantile regression lines geom_quantile to forecast in ggplot?
  • How to handle Sagemaker Batch Transform discarding a file with a failed model request
  • How to find a missing row between two similar 2d numpy arrays
  • Whether to clone an object for a prototype
  • How to make Kotlin Multiplatform work with Android Studio's project view?
  • Setting up multi-release JAR unit tests
Trending Discussions on preferable

QUESTION

Pandas DataFrame update cell values from second DataFrame

Asked 2022-Apr-07 at 09:49

Say I have two DataFrames: df1 and df2. df2 beeing a subframe of df1. Eg:

df1=pd.DataFrame(index=[0,1,2,3,4],columns=['a','b','c']).fillna(0)

df2=pd.DataFrame([1,2,3],index=[0,2,4],columns=['b'])

Is there a more elegant, implicit version of this:

df1.loc[df2.index,df2.columns]=df2.values

And why would it be preferable?

ANSWER

Answered 2022-Apr-07 at 09:49

You can use update:

df1.update(df2)

The operation is in place (no output), and might change the type of the data.

resulting df1:

   a    b  c
0  0  1.0  0
1  0  0.0  0
2  0  2.0  0
3  0  0.0  0
4  0  3.0  0

A workaround to help getting integer type is to use convert_dtypes on df2:

df1.update(df2.convert_dtypes())

output:

   a  b  c
0  0  1  0
1  0  0  0
2  0  2  0
3  0  0  0
4  0  3  0

Source https://stackoverflow.com/questions/71779894

Community Discussions, Code Snippets contain sources that include Stack Exchange Network

Vulnerabilities

No vulnerabilities reported

Install preferable

Add this line to your Gemfile:.

Support

For any new features, suggestions and bugs create an issue on GitHub. If you have any questions check and ask questions on community page Stack Overflow .

DOWNLOAD this Library from

Find, review, and download reusable Libraries, Code Snippets, Cloud APIs from
over 430 million Knowledge Items
Find more libraries
Reuse Solution Kits and Libraries Curated by Popular Use Cases

Save this library and start creating your kit

Explore Related Topics

Share this Page

share link
Find, review, and download reusable Libraries, Code Snippets, Cloud APIs from
over 430 million Knowledge Items
Find more libraries
Reuse Solution Kits and Libraries Curated by Popular Use Cases

Save this library and start creating your kit

  • © 2022 Open Weaver Inc.