Robust | Android HotFix solution with high compatibility

by Meituan-Dianping Java Version: 0.4.87 License: Apache-2.0

Download this library from

kandi X-RAY | Robust Summary

Robust is a Java library. Robust has no bugs, it has no vulnerabilities, it has build file available, it has a Permissive License and it has medium support. You can download it from GitHub.

Robust is an Android HotFix solution with high compatibility and high stability. Robust can fix bugs immediately without publishing apk. More help on Wiki.

Support

Quality

Security

License

Reuse

kandi-support Support

Robust has a medium active ecosystem.
It has 4019 star(s) with 714 fork(s). There are 152 watchers for this library.
It had no major release in the last 12 months.
There are 136 open issues and 225 have been closed. On average issues are closed in 39 days. There are 6 open pull requests and 0 closed requests.
It has a neutral sentiment in the developer community.
The latest version of Robust is 0.4.87

Robust Support

Best in #Java

Average in #Java

Robust Support

Best in #Java

Average in #Java

quality kandi Quality

Robust has 0 bugs and 0 code smells.

Robust Quality

Best in #Java

Average in #Java

Robust Quality

Best in #Java

Average in #Java

security Security

Robust has no vulnerabilities reported, and its dependent libraries have no vulnerabilities reported.
Robust code analysis shows 0 unresolved vulnerabilities.
There are 0 security hotspots that need review.

Robust Security

Best in #Java

Average in #Java

Robust Security

Best in #Java

Average in #Java

license License

Robust is licensed under the Apache-2.0 License. This license is Permissive.
Permissive licenses have the least restrictions, and you can use them in most projects.

Robust License

Best in #Java

Average in #Java

Robust License

Best in #Java

Average in #Java

build Reuse

Robust releases are available to install and integrate.
Build file is available. You can build the component from source.
Installation instructions are not available. Examples and code snippets are available.

Robust Reuse

Best in #Java

Average in #Java

Robust Reuse

Best in #Java

Average in #Java

Top functions reviewed by kandi - BETA

kandi has reviewed Robust and discovered the below as its top functions. This is intended to give you an instant insight into Robust implemented functionality, and help decide if they suit your requirements.

Returns the access method body
Verify a patch
handle smaliases
Determines if the given method is a call method .
Read mapping file .
Prints the member mapping info .
Create insert method .
Initializes the View .
Get declared method .
Create the patches info class .

Robust Key Features

Support 2.3 to 10 Android OS

Perfect compatibility

Patch takes effect without a reboot

Support fixing at method level,including static methods

Support add classes and methods

Support ProGuard,including inline methods or changing methods' signature

Robust Examples and Code Snippets

See all related Code Snippets

Usage

Copy

Download

	apply plugin: 'com.android.application'
	//please uncomment fellow line before you build a patch
	//apply plugin: 'auto-patch-plugin'
	apply plugin: 'robust'

	compile 'com.meituan.robust:robust:0.4.99'

Steps

Copy

Download

	 apply plugin: 'com.android.application'
	 apply plugin: 'auto-patch-plugin'

Demo Usage

Copy

Download

	./gradlew clean  assembleRelease --stacktrace --no-daemon

Attentions

Copy

Download

	method a(){
	  return this;
	}

License

Copy

Download

Copyright 2017 Meituan-Dianping

Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at

   http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.

Creating nested columns in python dataframe

Copy

Download

import pandas as pd

# I assume that you can read raw data named test.csv by pandas and
# set header = None cause you mentioned the Test data without any headers, so:
df = pd.read_csv("test.csv", header = None)

# Then define preferred Columns! 
MyColumns = pd.MultiIndex.from_tuples([("Models" , ""),
                                       ("Accuracy without normalization" , ""),
                                       ("Accuracy with normalization" , "zscore"),
                                       ("Accuracy with normalization" , "minmax"),
                                       ("Accuracy with normalization" , "maxabs"),
                                       ("Accuracy with normalization" , "robust")])

# Create new DataFrame with specified Columns, after this you should pass values 
New_DataFrame = pd.DataFrame(df , columns = MyColumns)

# a loop for passing values
for item in range(len(MyColumns)):
    New_DataFrame.loc[: , MyColumns[item]] = df.iloc[: , item]

New_DataFrame.set_index(New_DataFrame.columns[0][0] , inplace=True)
New_DataFrame

-----------------------

import pandas as pd

# I assume that you can read raw data named test.csv by pandas and
# set header = None cause you mentioned the Test data without any headers, so:
df = pd.read_csv("test.csv", header = None)

# Then define preferred Columns! 
MyColumns = pd.MultiIndex.from_tuples([("Models" , ""),
                                       ("Accuracy without normalization" , ""),
                                       ("Accuracy with normalization" , "zscore"),
                                       ("Accuracy with normalization" , "minmax"),
                                       ("Accuracy with normalization" , "maxabs"),
                                       ("Accuracy with normalization" , "robust")])

# Create new DataFrame with specified Columns, after this you should pass values 
New_DataFrame = pd.DataFrame(df , columns = MyColumns)

# a loop for passing values
for item in range(len(MyColumns)):
    New_DataFrame.loc[: , MyColumns[item]] = df.iloc[: , item]

New_DataFrame.set_index(New_DataFrame.columns[0][0] , inplace=True)
New_DataFrame

Discrepancy in Javascript Intl.DateTimeFormat() Outputs for the Islamic (Hijri) Calendar between 'islamic' and 'ar-SA'

Copy

Download

new Date('2022-03-03').toLocaleDateString('en-US');
// outputs '3/2/2022' when run in San Francisco

new Date('2022-03-03').toLocaleDateString('en-US', { timeZone: 'UTC' });
// outputs '3/3/2022' as expected. UTC is used both for declaring and formatting.

new Date('2022-03-03').toLocaleDateString(
  'en-SA-u-ca-islamic-umalqura',
  { timeZone: 'UTC', month: 'long', day: 'numeric', year: 'numeric' }
);
// outputs 'Rajab 30, 1443 AH' as expected

new Intl.DateTimeFormat('ar-SA').resolvedOptions();
// {
//   calendar: "islamic-umalqura"
//   day: "numeric"
//   locale: "ar-SA"
//   month: "numeric"
//   numberingSystem: "arab"
//   timeZone: "America/Los_Angeles"
//   year: "numeric"
// }

calendars = ["islamic", "islamic-umalqura", "islamic-tbla", "islamic-civil", "islamic-rgsa"];
options = { timeZone: 'UTC', month: 'long', day: 'numeric', year: 'numeric' };
date = new Date('2022-03-03');
calendars.forEach(calendar => {
  formatted = date.toLocaleDateString(`en-SA-u-ca-${calendar}`, options);
  console.log(`${calendar}: ${formatted}`);
});
// The code above outputs the following:
//   islamic: Shaʻban 1, 1443 AH
//   islamic-umalqura: Rajab 30, 1443 AH
//   islamic-tbla: Rajab 30, 1443 AH
//   islamic-civil: Rajab 29, 1443 AH
//   islamic-rgsa: Shaʻban 1, 1443 AH

-----------------------

new Date('2022-03-03').toLocaleDateString('en-US');
// outputs '3/2/2022' when run in San Francisco

new Date('2022-03-03').toLocaleDateString('en-US', { timeZone: 'UTC' });
// outputs '3/3/2022' as expected. UTC is used both for declaring and formatting.

new Date('2022-03-03').toLocaleDateString(
  'en-SA-u-ca-islamic-umalqura',
  { timeZone: 'UTC', month: 'long', day: 'numeric', year: 'numeric' }
);
// outputs 'Rajab 30, 1443 AH' as expected

new Intl.DateTimeFormat('ar-SA').resolvedOptions();
// {
//   calendar: "islamic-umalqura"
//   day: "numeric"
//   locale: "ar-SA"
//   month: "numeric"
//   numberingSystem: "arab"
//   timeZone: "America/Los_Angeles"
//   year: "numeric"
// }

calendars = ["islamic", "islamic-umalqura", "islamic-tbla", "islamic-civil", "islamic-rgsa"];
options = { timeZone: 'UTC', month: 'long', day: 'numeric', year: 'numeric' };
date = new Date('2022-03-03');
calendars.forEach(calendar => {
  formatted = date.toLocaleDateString(`en-SA-u-ca-${calendar}`, options);
  console.log(`${calendar}: ${formatted}`);
});
// The code above outputs the following:
//   islamic: Shaʻban 1, 1443 AH
//   islamic-umalqura: Rajab 30, 1443 AH
//   islamic-tbla: Rajab 30, 1443 AH
//   islamic-civil: Rajab 29, 1443 AH
//   islamic-rgsa: Shaʻban 1, 1443 AH

-----------------------

new Date('2022-03-03').toLocaleDateString('en-US');
// outputs '3/2/2022' when run in San Francisco

new Date('2022-03-03').toLocaleDateString('en-US', { timeZone: 'UTC' });
// outputs '3/3/2022' as expected. UTC is used both for declaring and formatting.

new Date('2022-03-03').toLocaleDateString(
  'en-SA-u-ca-islamic-umalqura',
  { timeZone: 'UTC', month: 'long', day: 'numeric', year: 'numeric' }
);
// outputs 'Rajab 30, 1443 AH' as expected

new Intl.DateTimeFormat('ar-SA').resolvedOptions();
// {
//   calendar: "islamic-umalqura"
//   day: "numeric"
//   locale: "ar-SA"
//   month: "numeric"
//   numberingSystem: "arab"
//   timeZone: "America/Los_Angeles"
//   year: "numeric"
// }

calendars = ["islamic", "islamic-umalqura", "islamic-tbla", "islamic-civil", "islamic-rgsa"];
options = { timeZone: 'UTC', month: 'long', day: 'numeric', year: 'numeric' };
date = new Date('2022-03-03');
calendars.forEach(calendar => {
  formatted = date.toLocaleDateString(`en-SA-u-ca-${calendar}`, options);
  console.log(`${calendar}: ${formatted}`);
});
// The code above outputs the following:
//   islamic: Shaʻban 1, 1443 AH
//   islamic-umalqura: Rajab 30, 1443 AH
//   islamic-tbla: Rajab 30, 1443 AH
//   islamic-civil: Rajab 29, 1443 AH
//   islamic-rgsa: Shaʻban 1, 1443 AH

-----------------------

============================================================
islamic diff from islamic-tbla           :6369  days (34.19%)
islamic diff from islamic-rgsa           :0     days ( 0.00%)
islamic diff from islamic-umalqura       :11425 days (61.33%)
islamic diff from islamic-civil          :15919 days (85.46%)
------------------------------------------------------------
islamic-tbla diff from islamic-rgsa      :6369  days (34.19%)
islamic-tbla diff from islamic-umalqura  :10777 days (57.85%)
islamic-tbla diff from islamic-civil     :18628 days (100.00%)
-------------------------------------------------------------
islamic-rgsa diff from islamic-umalqura  :11425 days (61.33%)
islamic-rgsa diff from islamic-civil     :15919 days (85.46%)
-------------------------------------------------------------
islamic-umalqura diff from islamic-civil :9049  days (48.58%)
=============================================================

console.log("Comparing 50 years results under the 5 islamic calendar formats (from year 1980 to 2030)");
console.log("=".repeat(60));

let startDate = new Date("1980-01-01");  // date to start test from
let options   = {year: 'numeric', month: 'long', day: 'numeric'};
let d         = startDate;
let diff      = [0,0,0,0,0,0,0,0,0,0];  // array to hold the diff results
let totalDays = 18628;                  // total days approx 50 Gregorian years
for (let i=0; i<totalDays; i++) {
let dateIslamic     = new Intl.DateTimeFormat('en-u-ca-islamic'         ,options).format(d),
    dateIslamicTBLA = new Intl.DateTimeFormat('en-u-ca-islamic-tbla'    ,options).format(d),
    dateIslamicRGSA = new Intl.DateTimeFormat('en-u-ca-islamic-rgsa'    ,options).format(d),
    dateIslamicUMQ  = new Intl.DateTimeFormat('en-u-ca-islamic-umalqura',options).format(d),
    dateIslamicCIVL = new Intl.DateTimeFormat('en-u-ca-islamic-civil'   ,options).format(d);

if (dateIslamic != dateIslamicTBLA) diff[0]++;
if (dateIslamic != dateIslamicRGSA) diff[1]++;
if (dateIslamic != dateIslamicUMQ)  diff[2]++;
if (dateIslamic != dateIslamicCIVL) diff[3]++;

if (dateIslamicTBLA != dateIslamicRGSA) diff[4]++;
if (dateIslamicTBLA != dateIslamicUMQ)  diff[5]++;
if (dateIslamicTBLA != dateIslamicCIVL) diff[6]++;

if (dateIslamicRGSA != dateIslamicUMQ)  diff[7]++;
if (dateIslamicRGSA != dateIslamicCIVL) diff[8]++;

if (dateIslamicUMQ != dateIslamicCIVL)  diff[9]++;

d    = new Date(d.setDate(d.getDate() + 1)); // next day
}

console.log("islamic diff from islamic-tbla           :"+perc(0));
console.log("islamic diff from islamic-rgsa           :"+perc(1));
console.log("islamic diff from islamic-umalqura       :"+perc(2));
console.log("islamic diff from islamic-civil          :"+perc(3));
console.log("-".repeat(50));
console.log("islamic-tbla diff from islamic-rgsa      :"+perc(4));
console.log("islamic-tbla diff from islamic-umalqura  :"+perc(5));
console.log("islamic-tbla diff from islamic-civil     :"+perc(6));
console.log("-".repeat(50));
console.log("islamic-rgsa diff from islamic-umalqura  :"+perc(7));
console.log("islamic-rgsa diff from islamic-civil     :"+perc(8));
console.log("-".repeat(50));
console.log("islamic-umalqura diff from islamic-civil :"+perc(9));


function perc(n) {return + diff[n]+" days (" +((diff[n]/totalDays)*100).toFixed(2)+"%)";}

-----------------------

============================================================
islamic diff from islamic-tbla           :6369  days (34.19%)
islamic diff from islamic-rgsa           :0     days ( 0.00%)
islamic diff from islamic-umalqura       :11425 days (61.33%)
islamic diff from islamic-civil          :15919 days (85.46%)
------------------------------------------------------------
islamic-tbla diff from islamic-rgsa      :6369  days (34.19%)
islamic-tbla diff from islamic-umalqura  :10777 days (57.85%)
islamic-tbla diff from islamic-civil     :18628 days (100.00%)
-------------------------------------------------------------
islamic-rgsa diff from islamic-umalqura  :11425 days (61.33%)
islamic-rgsa diff from islamic-civil     :15919 days (85.46%)
-------------------------------------------------------------
islamic-umalqura diff from islamic-civil :9049  days (48.58%)
=============================================================

console.log("Comparing 50 years results under the 5 islamic calendar formats (from year 1980 to 2030)");
console.log("=".repeat(60));

let startDate = new Date("1980-01-01");  // date to start test from
let options   = {year: 'numeric', month: 'long', day: 'numeric'};
let d         = startDate;
let diff      = [0,0,0,0,0,0,0,0,0,0];  // array to hold the diff results
let totalDays = 18628;                  // total days approx 50 Gregorian years
for (let i=0; i<totalDays; i++) {
let dateIslamic     = new Intl.DateTimeFormat('en-u-ca-islamic'         ,options).format(d),
    dateIslamicTBLA = new Intl.DateTimeFormat('en-u-ca-islamic-tbla'    ,options).format(d),
    dateIslamicRGSA = new Intl.DateTimeFormat('en-u-ca-islamic-rgsa'    ,options).format(d),
    dateIslamicUMQ  = new Intl.DateTimeFormat('en-u-ca-islamic-umalqura',options).format(d),
    dateIslamicCIVL = new Intl.DateTimeFormat('en-u-ca-islamic-civil'   ,options).format(d);

if (dateIslamic != dateIslamicTBLA) diff[0]++;
if (dateIslamic != dateIslamicRGSA) diff[1]++;
if (dateIslamic != dateIslamicUMQ)  diff[2]++;
if (dateIslamic != dateIslamicCIVL) diff[3]++;

if (dateIslamicTBLA != dateIslamicRGSA) diff[4]++;
if (dateIslamicTBLA != dateIslamicUMQ)  diff[5]++;
if (dateIslamicTBLA != dateIslamicCIVL) diff[6]++;

if (dateIslamicRGSA != dateIslamicUMQ)  diff[7]++;
if (dateIslamicRGSA != dateIslamicCIVL) diff[8]++;

if (dateIslamicUMQ != dateIslamicCIVL)  diff[9]++;

d    = new Date(d.setDate(d.getDate() + 1)); // next day
}

console.log("islamic diff from islamic-tbla           :"+perc(0));
console.log("islamic diff from islamic-rgsa           :"+perc(1));
console.log("islamic diff from islamic-umalqura       :"+perc(2));
console.log("islamic diff from islamic-civil          :"+perc(3));
console.log("-".repeat(50));
console.log("islamic-tbla diff from islamic-rgsa      :"+perc(4));
console.log("islamic-tbla diff from islamic-umalqura  :"+perc(5));
console.log("islamic-tbla diff from islamic-civil     :"+perc(6));
console.log("-".repeat(50));
console.log("islamic-rgsa diff from islamic-umalqura  :"+perc(7));
console.log("islamic-rgsa diff from islamic-civil     :"+perc(8));
console.log("-".repeat(50));
console.log("islamic-umalqura diff from islamic-civil :"+perc(9));


function perc(n) {return + diff[n]+" days (" +((diff[n]/totalDays)*100).toFixed(2)+"%)";}

Javascript: frame precise video stop

Copy

Download

   const callback = (now, metadata) => {
      if (startTime == 0) {
        startTime = now;
      }
      elapsed = metadata.mediaTime;
      currentFrame = metadata.presentedFrames - doneCount;

      fps = (currentFrame / elapsed).toFixed(3);
      fps = !isFinite(fps) ? 0 : fps;

      updateStats();
      if (stopFrames.includes(currentFrame)) {
        pauseMyVideo();
      } else {
        video.requestVideoFrameCallback(callback);
      }
   };
   video.requestVideoFrameCallback(callback);

-----------------------

<!DOCTYPE html>
<html>
<body>

<h1 style="position: absolute; top: 10px; left: 10px" > Demo // Stop Video at Specific Frame(s) : </h1>
<br>

<div style="z-index: 1; overflow:hidden; position: absolute; top: 60px; left: 10px; font-family: OpenSans; font-size: 14px">
<p> <b> Choose an .MP4 video file... </b> </p>
<input type="file" id="choose_media" accept=".mov, .mp4" />
</div>

<video id="myVideo" width="640" height="480" controls muted playsinline 
style="position: absolute; top: 80px; left: 10px" >
<source src="vc_timecode3.mp4" type="video/mp4">
</video>

<div id="cont_texts" style="position: absolute; top: 80px; left: 700px" >

<span> Current Time : </span> <span id="txt_curr_time"> 00:00:00 </span> 
<br><br>
<span> Estimated Frame Num : </span> <span id="txt_est_frame"> 0 </span> 
<br><br>
<span> Total Frames (video) : </span> <span id="txt_total_frame"> -999 </span> 
<br><br>

<span onclick="check_points()" > Stopping Points Array : </span> <input type="text" id="stopPointsArray" value="" > 

</div>

</body>


<script>


////////////////////////////////

//# VARS
var myVideo = document.getElementById( 'myVideo' );
var video_duration;

var h; var m; var s;
var h_time; var m_time; var s_time;

var vid_curr_time = document.getElementById( 'txt_curr_time' );
var vid_est_frame = document.getElementById( 'txt_est_frame' );
var vid_total_frame = document.getElementById( 'txt_total_frame' );

var reader; //to get bytes from file into Array
var bytes_MP4; //updated as Array

//# MP4 related vars
var got_FPS = false; var video_FPS = -1; 
var temp_Pos = 0;  var sampleCount = -1;
var temp_int_1, temp_int_2 = -1;

                    
var array_points = document.getElementById("stopPointsArray");
array_points.addEventListener('change', check_points );

//# EVENTS
document.getElementById('choose_media').addEventListener('change', onFileSelected, false);

myVideo.addEventListener("timeupdate", video_timeUpdate);           
myVideo.addEventListener("play", handle_Media_Events );
myVideo.addEventListener("pause", handle_Media_Events );
myVideo.addEventListener("ended", handle_Media_Events );

//# LET'S GO...
        
function onFileSelected ( evt )
{
    file = evt.target.files[0];
    path = (window.URL || window.webkitURL).createObjectURL(file);
    
    reader = new FileReader();
    reader.readAsArrayBuffer(file);
    
    
    reader.onloadend = function(evt) 
    {
        //alert( " file is selected ... " );
        
        if (evt.target.readyState == FileReader.DONE) 
        {
            bytes_MP4 = new Uint8Array( evt.target.result );
            get_MP4_info( bytes_MP4 );
            
            //# use bytes Array as video tag source
            let path_MP4 = (window.URL || window.webkitURL).createObjectURL(new Blob([bytes_MP4], { type: 'video/mp4' })); //'image/png' //'application/octet-stream'
            myVideo.src = path_MP4;
            myVideo.load();
            
            video_duration = myVideo.duration;
            txt_total_frame.innerHTML =( sampleCount);
            //alert("video FPS : " + video_FPS);
        }
        
    }
    
}

function check_points (e)
{
    alert( "Array Points are : " + e.target.value );
}

function handle_Media_Events()
{
    if ( event.type == "ended" )
    { 
        myVideo.currentTime = 0; myVideo.play(); myVideo.pause(); myVideo.play();
    }
    
    //{ myVideo.currentTime = 8; btn_ctrl.src = "ico_vc_play.png"; vid_isPlaying = false; bool_isPlaying = true; }
    
    if ( event.type == "play" )
    {
        if ( myVideo.nodeName == "VIDEO" )
        {

        }
    
    }
    
    else if ( event.type == "pause" )
    {
        
        
    }
    
    else if ( event.type == "seeked" )
    {
        
        
    }
    
}

function video_timeUpdate()
{
    vid_curr_time.innerHTML = ( convertTime ( myVideo.currentTime ) );
    
    vid_est_frame.innerHTML = Math.round ( video_FPS * myVideo.currentTime );
    
}

function convertTime ( input_Secs ) 
{
    h = Math.floor(input_Secs / 3600);
    m = Math.floor(input_Secs % 3600 / 60);
    s = Math.floor(input_Secs % 3600 % 60);

    h_time = h < 10 ? ("0" + h) : h ;
    m_time = m < 10 ? ("0" + m) : m ;
    s_time = s < 10 ? ("0" + s) : s ;
    
    if ( (h_time == 0) && ( video_duration < 3600) ) 
    { return ( m_time + ":" + s_time ); }
    else 
    { return ( h_time + ":" + m_time + ":" + s_time ); }
     
}

function get_MP4_info( input ) //# "input" is Array of Byte values
{
    //# console.log( "checking MP4 frame-rate..." );
    
    got_FPS = false;
    temp_Pos = 0; //# our position inside bytes of MP4 array
     
    let hdlr_type = "-1"; 
    
    while(true)
    {
        //# Step 1) Prepare for when metadata pieces are found  
        //# When VIDEO HANDLER Atom is found in MP4
        
        //# if STSZ ... 73 74 73 7A  
        if (input[ temp_Pos+0 ] == 0x73)
        {
            if ( ( input[temp_Pos+1] == 0x74 ) && ( input[temp_Pos+2] == 0x73 ) && ( input[temp_Pos+3] == 0x7A ) )
            {
                if ( hdlr_type == "vide" ) //# only IF it's the "video" track
                {
                    temp_Pos += 12;
                    sampleCount = ( ( input[temp_Pos+0] << 24) | (input[temp_Pos+1] << 16) | (input[temp_Pos+2] << 8) | input[temp_Pos+3] );
                    console.log( "found VIDEO sampleCount at: " + sampleCount );
                    
                    video_FPS = ( ( sampleCount * temp_int_1 ) / temp_int_2 );
                    console.log( "FPS of MP4 ### : " +  video_FPS );
                }
                
            }
            
        }
        
        //# Step 2) Find the pieces of metadata info
        //# Find other Atoms with data needed by above VIDEO HANDLER code.
        
        
        //# for MOOV and MDAT
        if (input[ temp_Pos ] == 0x6D) //0x6D
        {
            //# if MDAT ... 6D 64 61 74
            if ( ( temp_Pos[temp_Pos+1] == 0x64 ) && ( input[temp_Pos+2] == 0x61 ) && ( input[temp_Pos+3] == 0x74 ) )
            {
                temp_int = ( ( input[temp_Pos-4] << 24) | (input[temp_Pos-3] << 16) | (input[temp_Pos-2] << 8) | input[temp_Pos-1] );
                temp_Pos = (temp_int-1);
                if ( temp_Pos >= (input.length-1) ) { break; }
            }
            
            //# if MOOV ... 6D 6F 6F 76
            if ( ( input[temp_Pos+1] == 0x6F ) && ( input[temp_Pos+2] == 0x6F ) && ( input[temp_Pos+3] == 0x76 ) )
            {
                temp_int = ( ( input[temp_Pos-4] << 24) | (input[temp_Pos-3] << 16) | (input[temp_Pos-2] << 8) | input[temp_Pos-1] );
            }
            
            //# if MDHD ... 6D 64 68 64
            if ( ( input[temp_Pos+1] == 0x64 ) && ( input[temp_Pos+2] == 0x68 ) && ( input[temp_Pos+3] == 0x64 ) )
            {
                temp_Pos += 32;
                
                //# if HDLR ... 68 64 6C 72
                if (  input[temp_Pos+0] == 0x68 )
                {
                    if ( ( input[temp_Pos+1] == 0x64 ) && ( input[temp_Pos+2] == 0x6C ) && ( input[temp_Pos+3] == 0x72 ) )
                    {
                        temp_Pos += 12;
                        hdlr_type = String.fromCharCode(input[temp_Pos+0], input[temp_Pos+1], input[temp_Pos+2], input[temp_Pos+3] );
                    }
                }
            }
            
            //# if MVHD ... 6D 76 68 64
            if ( ( input[temp_Pos+1] == 0x76 ) && ( input[temp_Pos+2] == 0x68 ) && ( input[temp_Pos+3] == 0x64 ) )
            {
                temp_Pos += (12 + 4);
                
                //# get timescale
                temp_int_1 = ( ( input[temp_Pos+0] << 24) | (input[temp_Pos+1] << 16) | (input[temp_Pos+2] << 8) | input[temp_Pos+3] );
                ///console.log( "MVHD timescale at: " + temp_int_1 );
                
                //# get duration
                temp_int_2 = ( ( input[temp_Pos+4+0] << 24) | (input[temp_Pos+4+1] << 16) | (input[temp_Pos+4+2] << 8) | input[temp_Pos+4+3] );
                ///console.log( "MVHD duration at: " + temp_int_2 );
            }
            
        }
        
        if( temp_Pos >= (input.length-1) ) { break; }
        if( got_FPS == true) { break; }
        
        temp_Pos++;
    }
    
}

</script>

</html>

In the ompr package in R, how can I rephrase my objective/constraints/variables so as to avoid the "problem too large" error?

Copy

Download

require(slam)
boatsSTM<-as.simple_triplet_matrix(boats.n.ij)
...

#setting the objective function
set_objective(sum_expr(u[boatsSTM$i[k], boatsSTM$j[k]] * boatsSTM$v[k], k = 1:length(boatsSTM$i)))

Which form is better for return value of functions in embedded C?

Copy

Download

locvar = func();

if ((locvar == 1) || (locvar == 2)) {
  ...
}

Why does summary() show different standard errors than coeftest()?

Copy

Download

fit <- glm(am ~ hp + cyl, family=binomial(link="logit"), mtcars)

summary(fit)
#             Estimate Std. Error z value Pr(>|z|)   
# (Intercept)  5.83220    2.06595   2.823  0.00476 **
# hp           0.02775    0.01366   2.031  0.04228 * 
# cyl         -1.70306    0.60286  -2.825  0.00473 **

library(sandwich); library(lmtest)

coeftest(fit, vcov.=vcov(fit))
#              Estimate Std. Error z value Pr(>|z|)   
# (Intercept)  5.832204   2.065949  2.8230 0.004757 **
# hp           0.027748   0.013664  2.0307 0.042282 * 
# cyl         -1.703064   0.602862 -2.8250 0.004729 **

(ct <- coeftest(fit, vcov.=vcovHC(fit, type='HC0')))
#              Estimate Std. Error z value Pr(>|z|)   
# (Intercept)  5.832204   2.139307  2.7262 0.006407 **
# hp           0.027748   0.012254  2.2643 0.023553 * 
# cyl         -1.703064   0.572045 -2.9772 0.002909 **

texreg::screenreg(fit)
# =========================
#                 Model 1  
# -------------------------
# (Intercept)       5.83 **
#                  (2.07)  
# hp                0.03 * 
#                  (0.01)  
# cyl              -1.70 **
#                  (0.60)  
# -------------------------
# AIC              34.63   
# BIC              39.02   
# Log Likelihood  -14.31   
# Deviance         28.63   
# Num. obs.        32      
# =========================
# *** p < 0.001; ** p < 0.01; * p < 0.05

texreg::screenreg(fit, 
                  override.pvalues=ct[, 4],
                  override.se=ct[, 3])
# =========================
#                 Model 1  
# -------------------------
# (Intercept)       5.83 **
#                  (2.73)  
# hp                0.03 * 
#                  (2.26)  
# cyl              -1.70 **
#                 (-2.98)  
# -------------------------
# AIC              34.63   
# BIC              39.02   
# Log Likelihood  -14.31   
# Deviance         28.63   
# Num. obs.        32      
# =========================
# *** p < 0.001; ** p < 0.01; * p < 0.05

-----------------------

fit <- glm(am ~ hp + cyl, family=binomial(link="logit"), mtcars)

summary(fit)
#             Estimate Std. Error z value Pr(>|z|)   
# (Intercept)  5.83220    2.06595   2.823  0.00476 **
# hp           0.02775    0.01366   2.031  0.04228 * 
# cyl         -1.70306    0.60286  -2.825  0.00473 **

library(sandwich); library(lmtest)

coeftest(fit, vcov.=vcov(fit))
#              Estimate Std. Error z value Pr(>|z|)   
# (Intercept)  5.832204   2.065949  2.8230 0.004757 **
# hp           0.027748   0.013664  2.0307 0.042282 * 
# cyl         -1.703064   0.602862 -2.8250 0.004729 **

(ct <- coeftest(fit, vcov.=vcovHC(fit, type='HC0')))
#              Estimate Std. Error z value Pr(>|z|)   
# (Intercept)  5.832204   2.139307  2.7262 0.006407 **
# hp           0.027748   0.012254  2.2643 0.023553 * 
# cyl         -1.703064   0.572045 -2.9772 0.002909 **

texreg::screenreg(fit)
# =========================
#                 Model 1  
# -------------------------
# (Intercept)       5.83 **
#                  (2.07)  
# hp                0.03 * 
#                  (0.01)  
# cyl              -1.70 **
#                  (0.60)  
# -------------------------
# AIC              34.63   
# BIC              39.02   
# Log Likelihood  -14.31   
# Deviance         28.63   
# Num. obs.        32      
# =========================
# *** p < 0.001; ** p < 0.01; * p < 0.05

texreg::screenreg(fit, 
                  override.pvalues=ct[, 4],
                  override.se=ct[, 3])
# =========================
#                 Model 1  
# -------------------------
# (Intercept)       5.83 **
#                  (2.73)  
# hp                0.03 * 
#                  (2.26)  
# cyl              -1.70 **
#                 (-2.98)  
# -------------------------
# AIC              34.63   
# BIC              39.02   
# Log Likelihood  -14.31   
# Deviance         28.63   
# Num. obs.        32      
# =========================
# *** p < 0.001; ** p < 0.01; * p < 0.05

-----------------------

fit <- glm(am ~ hp + cyl, family=binomial(link="logit"), mtcars)

summary(fit)
#             Estimate Std. Error z value Pr(>|z|)   
# (Intercept)  5.83220    2.06595   2.823  0.00476 **
# hp           0.02775    0.01366   2.031  0.04228 * 
# cyl         -1.70306    0.60286  -2.825  0.00473 **

library(sandwich); library(lmtest)

coeftest(fit, vcov.=vcov(fit))
#              Estimate Std. Error z value Pr(>|z|)   
# (Intercept)  5.832204   2.065949  2.8230 0.004757 **
# hp           0.027748   0.013664  2.0307 0.042282 * 
# cyl         -1.703064   0.602862 -2.8250 0.004729 **

(ct <- coeftest(fit, vcov.=vcovHC(fit, type='HC0')))
#              Estimate Std. Error z value Pr(>|z|)   
# (Intercept)  5.832204   2.139307  2.7262 0.006407 **
# hp           0.027748   0.012254  2.2643 0.023553 * 
# cyl         -1.703064   0.572045 -2.9772 0.002909 **

texreg::screenreg(fit)
# =========================
#                 Model 1  
# -------------------------
# (Intercept)       5.83 **
#                  (2.07)  
# hp                0.03 * 
#                  (0.01)  
# cyl              -1.70 **
#                  (0.60)  
# -------------------------
# AIC              34.63   
# BIC              39.02   
# Log Likelihood  -14.31   
# Deviance         28.63   
# Num. obs.        32      
# =========================
# *** p < 0.001; ** p < 0.01; * p < 0.05

texreg::screenreg(fit, 
                  override.pvalues=ct[, 4],
                  override.se=ct[, 3])
# =========================
#                 Model 1  
# -------------------------
# (Intercept)       5.83 **
#                  (2.73)  
# hp                0.03 * 
#                  (2.26)  
# cyl              -1.70 **
#                 (-2.98)  
# -------------------------
# AIC              34.63   
# BIC              39.02   
# Log Likelihood  -14.31   
# Deviance         28.63   
# Num. obs.        32      
# =========================
# *** p < 0.001; ** p < 0.01; * p < 0.05

-----------------------

fit <- glm(am ~ hp + cyl, family=binomial(link="logit"), mtcars)

summary(fit)
#             Estimate Std. Error z value Pr(>|z|)   
# (Intercept)  5.83220    2.06595   2.823  0.00476 **
# hp           0.02775    0.01366   2.031  0.04228 * 
# cyl         -1.70306    0.60286  -2.825  0.00473 **

library(sandwich); library(lmtest)

coeftest(fit, vcov.=vcov(fit))
#              Estimate Std. Error z value Pr(>|z|)   
# (Intercept)  5.832204   2.065949  2.8230 0.004757 **
# hp           0.027748   0.013664  2.0307 0.042282 * 
# cyl         -1.703064   0.602862 -2.8250 0.004729 **

(ct <- coeftest(fit, vcov.=vcovHC(fit, type='HC0')))
#              Estimate Std. Error z value Pr(>|z|)   
# (Intercept)  5.832204   2.139307  2.7262 0.006407 **
# hp           0.027748   0.012254  2.2643 0.023553 * 
# cyl         -1.703064   0.572045 -2.9772 0.002909 **

texreg::screenreg(fit)
# =========================
#                 Model 1  
# -------------------------
# (Intercept)       5.83 **
#                  (2.07)  
# hp                0.03 * 
#                  (0.01)  
# cyl              -1.70 **
#                  (0.60)  
# -------------------------
# AIC              34.63   
# BIC              39.02   
# Log Likelihood  -14.31   
# Deviance         28.63   
# Num. obs.        32      
# =========================
# *** p < 0.001; ** p < 0.01; * p < 0.05

texreg::screenreg(fit, 
                  override.pvalues=ct[, 4],
                  override.se=ct[, 3])
# =========================
#                 Model 1  
# -------------------------
# (Intercept)       5.83 **
#                  (2.73)  
# hp                0.03 * 
#                  (2.26)  
# cyl              -1.70 **
#                 (-2.98)  
# -------------------------
# AIC              34.63   
# BIC              39.02   
# Log Likelihood  -14.31   
# Deviance         28.63   
# Num. obs.        32      
# =========================
# *** p < 0.001; ** p < 0.01; * p < 0.05

-----------------------

fit <- glm(am ~ hp + cyl, family=binomial(link="logit"), mtcars)

summary(fit)
#             Estimate Std. Error z value Pr(>|z|)   
# (Intercept)  5.83220    2.06595   2.823  0.00476 **
# hp           0.02775    0.01366   2.031  0.04228 * 
# cyl         -1.70306    0.60286  -2.825  0.00473 **

library(sandwich); library(lmtest)

coeftest(fit, vcov.=vcov(fit))
#              Estimate Std. Error z value Pr(>|z|)   
# (Intercept)  5.832204   2.065949  2.8230 0.004757 **
# hp           0.027748   0.013664  2.0307 0.042282 * 
# cyl         -1.703064   0.602862 -2.8250 0.004729 **

(ct <- coeftest(fit, vcov.=vcovHC(fit, type='HC0')))
#              Estimate Std. Error z value Pr(>|z|)   
# (Intercept)  5.832204   2.139307  2.7262 0.006407 **
# hp           0.027748   0.012254  2.2643 0.023553 * 
# cyl         -1.703064   0.572045 -2.9772 0.002909 **

texreg::screenreg(fit)
# =========================
#                 Model 1  
# -------------------------
# (Intercept)       5.83 **
#                  (2.07)  
# hp                0.03 * 
#                  (0.01)  
# cyl              -1.70 **
#                  (0.60)  
# -------------------------
# AIC              34.63   
# BIC              39.02   
# Log Likelihood  -14.31   
# Deviance         28.63   
# Num. obs.        32      
# =========================
# *** p < 0.001; ** p < 0.01; * p < 0.05

texreg::screenreg(fit, 
                  override.pvalues=ct[, 4],
                  override.se=ct[, 3])
# =========================
#                 Model 1  
# -------------------------
# (Intercept)       5.83 **
#                  (2.73)  
# hp                0.03 * 
#                  (2.26)  
# cyl              -1.70 **
#                 (-2.98)  
# -------------------------
# AIC              34.63   
# BIC              39.02   
# Log Likelihood  -14.31   
# Deviance         28.63   
# Num. obs.        32      
# =========================
# *** p < 0.001; ** p < 0.01; * p < 0.05

Is mass assignment protection needed when properly using validation?

Copy

Download

if(isset($request->forbidden_param))

//in controller
MyModel::update($request->input())

// in model

$fillable = [ ... ]
//or
$guarded = [ ... ]

Model::update([
    'permitted_param1' => $request->permitted_param
    'permitted_param2' => $request->permitted_param2
]);

$guarded = [];

User::create($request->except('password') + [ 'secret' => bcrypt($request->password)] );

MyModel::update($request->safe()->all());

MyModel::update($request->validated())

-----------------------

if(isset($request->forbidden_param))

//in controller
MyModel::update($request->input())

// in model

$fillable = [ ... ]
//or
$guarded = [ ... ]

Model::update([
    'permitted_param1' => $request->permitted_param
    'permitted_param2' => $request->permitted_param2
]);

$guarded = [];

User::create($request->except('password') + [ 'secret' => bcrypt($request->password)] );

MyModel::update($request->safe()->all());

MyModel::update($request->validated())

-----------------------

if(isset($request->forbidden_param))

//in controller
MyModel::update($request->input())

// in model

$fillable = [ ... ]
//or
$guarded = [ ... ]

Model::update([
    'permitted_param1' => $request->permitted_param
    'permitted_param2' => $request->permitted_param2
]);

$guarded = [];

User::create($request->except('password') + [ 'secret' => bcrypt($request->password)] );

MyModel::update($request->safe()->all());

MyModel::update($request->validated())

-----------------------

if(isset($request->forbidden_param))

//in controller
MyModel::update($request->input())

// in model

$fillable = [ ... ]
//or
$guarded = [ ... ]

Model::update([
    'permitted_param1' => $request->permitted_param
    'permitted_param2' => $request->permitted_param2
]);

$guarded = [];

User::create($request->except('password') + [ 'secret' => bcrypt($request->password)] );

MyModel::update($request->safe()->all());

MyModel::update($request->validated())

-----------------------

if(isset($request->forbidden_param))

//in controller
MyModel::update($request->input())

// in model

$fillable = [ ... ]
//or
$guarded = [ ... ]

Model::update([
    'permitted_param1' => $request->permitted_param
    'permitted_param2' => $request->permitted_param2
]);

$guarded = [];

User::create($request->except('password') + [ 'secret' => bcrypt($request->password)] );

MyModel::update($request->safe()->all());

MyModel::update($request->validated())

-----------------------

if(isset($request->forbidden_param))

//in controller
MyModel::update($request->input())

// in model

$fillable = [ ... ]
//or
$guarded = [ ... ]

Model::update([
    'permitted_param1' => $request->permitted_param
    'permitted_param2' => $request->permitted_param2
]);

$guarded = [];

User::create($request->except('password') + [ 'secret' => bcrypt($request->password)] );

MyModel::update($request->safe()->all());

MyModel::update($request->validated())

-----------------------

if(isset($request->forbidden_param))

//in controller
MyModel::update($request->input())

// in model

$fillable = [ ... ]
//or
$guarded = [ ... ]

Model::update([
    'permitted_param1' => $request->permitted_param
    'permitted_param2' => $request->permitted_param2
]);

$guarded = [];

User::create($request->except('password') + [ 'secret' => bcrypt($request->password)] );

MyModel::update($request->safe()->all());

MyModel::update($request->validated())

How can I specialize std::common_type<A,B> so that it's naturally commutative?

Copy

Download

// define concept of `common_type<A,B>` existing
template <typename A, typename B>
concept has_in_common_ordered = requires { common_type<A,B>::type; };

namespace std {
    
    // define common_type<A,B> if common_type<B,A>::type exists:
    template <typename A, has_in_common_ordered<A> B>
    struct common_type<A,B> : public common_type<B,A> {};
    
}

struct X {};
struct Y {};

namespace std {
    
    template<>
    struct common_type<X,Y> {
        using type = X; // or whatever
    };
    
}

int main() {
    // even though we only specialized for one ordering,
    // both orderings now exist:
    std::common_type<X,Y>::type a;
    std::common_type<Y,X>::type b;
}

-----------------------

// define concept of `common_type<A,B>` existing
template <typename A, typename B>
concept has_in_common_ordered = requires { common_type<A,B>::type; };

namespace std {
    
    // define common_type<A,B> if common_type<B,A>::type exists:
    template <typename A, has_in_common_ordered<A> B>
    struct common_type<A,B> : public common_type<B,A> {};
    
}

struct X {};
struct Y {};

namespace std {
    
    template<>
    struct common_type<X,Y> {
        using type = X; // or whatever
    };
    
}

int main() {
    // even though we only specialized for one ordering,
    // both orderings now exist:
    std::common_type<X,Y>::type a;
    std::common_type<Y,X>::type b;
}

Periodically restart Python multiprocessing pool

Copy

Download

from multiprocessing import Pool, TimeoutError, cpu_count
from time import sleep
from random import randint


def log():
    print("logging is a dangerous activity: wear a hard hat.")


def work(d):
    sleep(randint(1, 100) / 100)
    print("finished working")
    if randint(1, 10) == 1:
        print("blocking...")
        while True:
            sleep(0.1)

    return d

data = list(range(100))
nproc = cpu_count()

while data:
    print(f"== Processing {len(data)} items. ==")
    with Pool(nproc) as p:
        tasks = [p.apply_async(work, (d,)) for d in data]

        for task in tasks:
            try:
                res = task.get(timeout=1)
                data.remove(res)
                log()
            except TimeoutError:
                failed.append(task)
                if len(failed) < nproc:
                    print(
                        f"{len(failed)} processes are blocked,"
                        f" but {nproc - len(failed)} remain."
                    )
                else:
                    break

            for task in failed:
                try:
                    res = task.get(timeout=0.01)
                    data.remove(res)
                    failed.remove(task)
                    log()
                except TimeoutError:
                    continue

-----------------------

from multiprocessing import Pool, TimeoutError, cpu_count
from time import sleep
from random import randint


def log():
    print("logging is a dangerous activity: wear a hard hat.")


def work(d):
    sleep(randint(1, 100) / 100)
    print("finished working")
    if randint(1, 10) == 1:
        print("blocking...")
        while True:
            sleep(0.1)

    return d

data = list(range(100))
nproc = cpu_count()

while data:
    print(f"== Processing {len(data)} items. ==")
    with Pool(nproc) as p:
        tasks = [p.apply_async(work, (d,)) for d in data]

        for task in tasks:
            try:
                res = task.get(timeout=1)
                data.remove(res)
                log()
            except TimeoutError:
                failed.append(task)
                if len(failed) < nproc:
                    print(
                        f"{len(failed)} processes are blocked,"
                        f" but {nproc - len(failed)} remain."
                    )
                else:
                    break

            for task in failed:
                try:
                    res = task.get(timeout=0.01)
                    data.remove(res)
                    failed.remove(task)
                    log()
                except TimeoutError:
                    continue

-----------------------

from multiprocessing import Pool, TimeoutError, cpu_count
from time import sleep
from random import randint


def log():
    print("logging is a dangerous activity: wear a hard hat.")


def work(d):
    sleep(randint(1, 100) / 100)
    print("finished working")
    if randint(1, 10) == 1:
        print("blocking...")
        while True:
            sleep(0.1)

    return d

data = list(range(100))
nproc = cpu_count()

while data:
    print(f"== Processing {len(data)} items. ==")
    with Pool(nproc) as p:
        tasks = [p.apply_async(work, (d,)) for d in data]

        for task in tasks:
            try:
                res = task.get(timeout=1)
                data.remove(res)
                log()
            except TimeoutError:
                failed.append(task)
                if len(failed) < nproc:
                    print(
                        f"{len(failed)} processes are blocked,"
                        f" but {nproc - len(failed)} remain."
                    )
                else:
                    break

            for task in failed:
                try:
                    res = task.get(timeout=0.01)
                    data.remove(res)
                    failed.remove(task)
                    log()
                except TimeoutError:
                    continue

-----------------------

from multiprocessing import Pool, TimeoutError, cpu_count
from time import sleep
from random import randint


def log():
    print("logging is a dangerous activity: wear a hard hat.")


def work(d):
    sleep(randint(1, 100) / 100)
    print("finished working")
    if randint(1, 10) == 1:
        print("blocking...")
        while True:
            sleep(0.1)

    return d

data = list(range(100))
nproc = cpu_count()

while data:
    print(f"== Processing {len(data)} items. ==")
    with Pool(nproc) as p:
        tasks = [p.apply_async(work, (d,)) for d in data]

        for task in tasks:
            try:
                res = task.get(timeout=1)
                data.remove(res)
                log()
            except TimeoutError:
                failed.append(task)
                if len(failed) < nproc:
                    print(
                        f"{len(failed)} processes are blocked,"
                        f" but {nproc - len(failed)} remain."
                    )
                else:
                    break

            for task in failed:
                try:
                    res = task.get(timeout=0.01)
                    data.remove(res)
                    failed.remove(task)
                    log()
                except TimeoutError:
                    continue

-----------------------

from multiprocessing import Pool, TimeoutError, cpu_count
from time import sleep
from random import randint


def log():
    print("logging is a dangerous activity: wear a hard hat.")


def work(d):
    sleep(randint(1, 100) / 100)
    print("finished working")
    if randint(1, 10) == 1:
        print("blocking...")
        while True:
            sleep(0.1)

    return d

data = list(range(100))
nproc = cpu_count()

while data:
    print(f"== Processing {len(data)} items. ==")
    with Pool(nproc) as p:
        tasks = [p.apply_async(work, (d,)) for d in data]

        for task in tasks:
            try:
                res = task.get(timeout=1)
                data.remove(res)
                log()
            except TimeoutError:
                failed.append(task)
                if len(failed) < nproc:
                    print(
                        f"{len(failed)} processes are blocked,"
                        f" but {nproc - len(failed)} remain."
                    )
                else:
                    break

            for task in failed:
                try:
                    res = task.get(timeout=0.01)
                    data.remove(res)
                    failed.remove(task)
                    log()
                except TimeoutError:
                    continue

How to use pytest to simulate full reboot

Copy

Download

import os
import time
import signal
import multiprocessing


class MyClass(multiprocessing.Process):
    def run(self):
        # Ping localhost for a limited amount of time
        os.system("ping -c 12 127.0.0.1")


process = MyClass()
process.start()

time.sleep(4)

print("terminating early")

# Send SIGKILL signal to the entire process group
group_id = os.getpgid(process.pid)
os.killpg(group_id, signal.SIGKILL)

print("done")

import os
import time
import multiprocessing

import psutil


class MyClass(multiprocessing.Process):
    def run(self):
        # Ping localhost for a limited amount of time
        os.system("ping -c 12 127.0.0.1")


def kill_process_group(pid):
    process = psutil.Process(pid)
    children = process.children(recursive=True)

    # First terminate all children
    for child in children:
        child.kill()
    psutil.wait_procs(children)

    # Then terminate the parent process
    process.kill()
    process.wait()


process = MyClass()
process.start()

time.sleep(4)

print("terminating early")

kill_process_group(process.pid)

print("done")

-----------------------

import os
import time
import signal
import multiprocessing


class MyClass(multiprocessing.Process):
    def run(self):
        # Ping localhost for a limited amount of time
        os.system("ping -c 12 127.0.0.1")


process = MyClass()
process.start()

time.sleep(4)

print("terminating early")

# Send SIGKILL signal to the entire process group
group_id = os.getpgid(process.pid)
os.killpg(group_id, signal.SIGKILL)

print("done")

import os
import time
import multiprocessing

import psutil


class MyClass(multiprocessing.Process):
    def run(self):
        # Ping localhost for a limited amount of time
        os.system("ping -c 12 127.0.0.1")


def kill_process_group(pid):
    process = psutil.Process(pid)
    children = process.children(recursive=True)

    # First terminate all children
    for child in children:
        child.kill()
    psutil.wait_procs(children)

    # Then terminate the parent process
    process.kill()
    process.wait()


process = MyClass()
process.start()

time.sleep(4)

print("terminating early")

kill_process_group(process.pid)

print("done")

See all related Code Snippets

Community Discussions

Trending Discussions on Robust

Creating nested columns in python dataframe
GitLab Runner fails to upload artifacts with "invalid argument" error
Discrepancy in Javascript Intl.DateTimeFormat() Outputs for the Islamic (Hijri) Calendar between 'islamic' and 'ar-SA'
Javascript: frame precise video stop
In the ompr package in R, how can I rephrase my objective/constraints/variables so as to avoid the "problem too large" error?
Which form is better for return value of functions in embedded C?
A clarification on the named requirements for containers
Why does summary() show different standard errors than coeftest()?
Is mass assignment protection needed when properly using validation?
How can I specialize std::common_type<A,B> so that it's naturally commutative?

Trending Discussions on Robust

QUESTION

Creating nested columns in python dataframe

Asked 2022-Feb-20 at 15:56

I have 3 columns namely Models(should be taken as index), Accuracy without normalization, Accuracy with normalization (zscore, minmax, maxabs, robust) and these are required to be created as:

 ------------------------------------------------------------------------------------
|   Models  |  Accuracy without normalization    |      Accuracy with normalization  |
|           |                                    |-----------------------------------|
|           |                                    | zscore | minmax | maxabs | robust |
 ------------------------------------------------------------------------------------

dfmod-> Models column
dfacc-> Accuracy without normalization
dfacc1-> Accuracy with normalization - zscore
dfacc2-> Accuracy with normalization - minmax
dfacc3-> Accuracy with normalization - maxabs
dfacc4-> Accuracy with normalization - robust

dfout=pd.DataFrame({('Accuracy without Normalization'):{dfacc},
     ('Accuracy using Normalization','zscore'):{dfacc1},
     ('Accuracy using Normalization','minmax'):{dfacc2},
     ('Accuracy using Normalization','maxabs'):{dfacc3},
     ('Accuracy using Normalization','robust'):{dfacc4},
   },index=dfmod
)

I was trying to do something like this but i can't figure out any further

Test data:

qda    0.6333       0.6917      0.5917      0.6417     0.5833
svm    0.5333       0.6917      0.5333      0.575      0.575
lda    0.5333       0.6583      0.5333      0.5667     0.5667
lr     0.5333       0.65        0.4917      0.5667     0.5667
dt     0.5333       0.65        0.4917      0.5667     0.5667
rc     0.5083       0.6333      0.4917      0.525      0.525
nb     0.5          0.625       0.475       0.5        0.4833
rfc    0.5          0.625       0.4417      0.4917     0.4583
knn    0.3917       0.6         0.4417      0.4833     0.45
et     0.375        0.5333      0.4333      0.4667     0.45
dc     0.375        0.5333      0.4333      0.4667     0.425
qds    0.3417       0.5333      0.4         0.4583     0.3667
lgt    0.3417       0.525       0.3917      0.45       0.3583
lt     0.2333       0.45        0.3917      0.4167     0.3417

These are values for respective subcolumns in order specified in the table above

ANSWER

Answered 2022-Feb-20 at 13:01

There's a dirty way to do this, I'll write about it till someone answers with a better idea. Here we go:

import pandas as pd

# I assume that you can read raw data named test.csv by pandas and
# set header = None cause you mentioned the Test data without any headers, so:
df = pd.read_csv("test.csv", header = None)

# Then define preferred Columns! 
MyColumns = pd.MultiIndex.from_tuples([("Models" , ""),
                                       ("Accuracy without normalization" , ""),
                                       ("Accuracy with normalization" , "zscore"),
                                       ("Accuracy with normalization" , "minmax"),
                                       ("Accuracy with normalization" , "maxabs"),
                                       ("Accuracy with normalization" , "robust")])

# Create new DataFrame with specified Columns, after this you should pass values 
New_DataFrame = pd.DataFrame(df , columns = MyColumns)

# a loop for passing values
for item in range(len(MyColumns)):
    New_DataFrame.loc[: , MyColumns[item]] = df.iloc[: , item]

This gives me:

after all, if you want to set Models as the index of New_DataFrame, You can continue with:

New_DataFrame.set_index(New_DataFrame.columns[0][0] , inplace=True)
New_DataFrame

This gives me:

Source https://stackoverflow.com/questions/71193085

QUESTION

GitLab Runner fails to upload artifacts with "invalid argument" error

Asked 2022-Feb-08 at 15:22

I'm completely new to trying to implement GitLab's CI/CD pipelines, but it's been going quite well. In fact, for my ASP.NET project, if I specify a Publish Profile in the msbuild command that uses Web Deploy, it actually deploys the code successfully to the web server.

However, I'm now wanting to have the "build" job create artifacts which are uploaded to GitLab that I can then subsequently deploy. We're using a self-hosted instance of GitLab, for which I'm not an admin, but I can speak to the admin if I know what I'm asking for!

So I've configured my gitlab-ci.yml file like this:

variables:
  NUGET_PATH: 'C:\Program Files\Nuget\Nuget.exe'
  NUGET_SOURCES: 'https://api.nuget.org/v3/index.json'
  MSBUILD_PATH: 'C:\Program Files (x86)\Microsoft Visual Studio\2022\BuildTools\MSBuild\Current\Bin\msbuild.exe'

stages:
- build

build-job:
  variables:
    CI_DEBUG_TRACE: "true"
  stage: build
  script:
    - '& "$env:NUGET_PATH" restore ApplicationTemplate.sln  -Source "$env:NUGET_SOURCES"'
    - '& "$env:MSBUILD_PATH" ApplicationTemplate\ApplicationTemplate.csproj /p:DeployOnBuild=true /p:Configuration=Release /p:PublishProfile=FolderPublish.pubxml'
  artifacts:
    paths: 
      - '.\ApplicationTemplate\bin\Release\Publish\'

The output shows that this builds the code just fine, and it also seems to successfully find the artifacts for upload. However, when it uploads the artifacts, even though the request gets a 200 OK response, the process fails. Here is the log output:

So, it finds the artifacts, it attempts to upload them and even gets a 200 OK response (in contrast to the handful of similar reports of this error I've been able to find online), but it still fails due to an invalid argument.

I've already enabled verbose debugging, as you can see from the output, but I'm none the wiser. Looking at the GitLab Runner entries in the Windows Event Log on the box where the runner is hosted doesn't shed any light on things either. The total size of the artifacts is 61.1MB, so I don't think my issue is related to that.

Can anyone see from this output what's invalid? Can I identify which argument is invalid and/or why it's invalid?

Edit: Things I've tried

Specifying a value for artifacts:expire_in.
Setting artifacts:public to FALSE, since I'm using a self-hosted GitLab environment and the default value for this setting (TRUE) is not valid in such an environment.
Trying every format I can think of for the value of the artifacts:paths setting (this seems to be incredibly robust - regardless of the format I use, the Runner seems to have no problem parsing it and finding the files to upload).
Taking a cue from this question I created a new project with a very simple build job to upload a single file:

stages:
  - build

build-job:
  variables:
    CI_DEBUG_TRACE: "true"
  stage: build
  script:
    - echo "Test" > test.txt
  artifacts:
    paths:
      - test.txt

About 50% of the time this job hangs on the uploading of the artifacts and I have to cancel it. The other half of the time it fails in exactly the same way as the my previous project:

ANSWER

Answered 2022-Feb-08 at 15:22

After countless hours working on this, it seems that ultimately the issue was that our internal Web Application Firewall was blocking some part of the transfer of artefacts to the server, or the response back from it. With the WAF reconfigured not to block traffic from the machine running the GitLab Runner, the artefacts are successfully uploaded and the job succeeds.

This would have been significantly easier to diagnose if the logging from GitLab was better. As per my comment on this issue, it should be possible to see the content of the response from the GitLab server after uploading artefacts, even when the response code is 200.

What's strange - and made diagnosing the issue even harder - is that when I worked through the issue with the admin of our GitLab instance, digging through logs and running it in debug mode, the artefact upload process was uploading something successfully. We could see, for example, the GitLab Runner's log had been uploaded to the server. Clearly the WAF's blocking was selective and didn't block everything in both directions.

Source https://stackoverflow.com/questions/70880840

QUESTION

Discrepancy in Javascript Intl.DateTimeFormat() Outputs for the Islamic (Hijri) Calendar between 'islamic' and 'ar-SA'

Asked 2022-Feb-06 at 13:50

The 3rd March 2022 is the end of this Hijri Month (month of Rajab for this year 1443 AH); i.e. 30 Rajab 1443 AH.

The month of Rajab for the year 1443 AH is 30 days in accordance with the Islamic (Hijri) Calendar in accordance with all websites, applications, MS Office, and Windows calendars.

When using the javascript Intl.DateTimeFormat() to display the Islamic (Hijri) date for the 3 March 2022 using the islamic calendar option, it will give the Islamic Hijri Date of (1 Shaʻban 1443 AH). This result is one day after the month of Rajab (i.e. the 1st of the following month) and it calculated the month Rajab to be 29 days rather than 30 days.

However, if the option passed to the Intl.DateTimeFormat() is ar-SA (i.e. arabic-Saudi Arabia), it will give the correct result. This is strange because the ar-SA locale uses the Islamic (Hijri) calendar by default.

Is this an error/bug or is it the correct internal workings of javascript?

Is there a more robust method to get the Islamic Date in Javascript other than using the 'ar-SA' locale (but not using external libraries)?

See the code example below:

I have tested this in node and chrome and it gives the same resulting discrepancy.

let date = new Date("2022-03-03");
let options = {year:'numeric', month:'long',day:'numeric'};

let format = new Intl.DateTimeFormat('ar-SA-u-nu-latn', options);
console.log("3 March 2022 with 'ar-SA'         :"+format.format(date)+ " ==> means: 30 Rajab 1443 AH");

format = new Intl.DateTimeFormat('en-u-ca-islamic-nu-latn', options);
console.log("3 March 2022 with Islamic Calendar: "+format.format(date));

ANSWER

Answered 2022-Feb-06 at 07:29

There are three possible reasons for the "off by one" date problems you're seeing:

Time zone mismatch between date initialization and date formatting
Using the wrong variation of the Islamic calendar (JS implementations typically offer 5 different Islamic calendars!)
Bugs in the ICU library used for JS's calendar calculations

I'll cover each of these below.

1. Time zone mismatch between date initialization and date formatting

The most common reason for off-by-one-day errors is (as @RobG noted in his comments above) a mismatch between the time zone used when declaring the Date value and the time zone used when formatting it in your desired calendar.

When you initialize a Date instance using an ISO 8601 string, the actual value stored in the Date instance is the number of milliseconds since January 1, 1970 UTC. Depending on your system time zone, new Date('2022-02-03') can be February 3 or February 2 in your system time zone. One way to evade this problem is to use UTC when formatting too:

new Date('2022-03-03').toLocaleDateString('en-US');
// outputs '3/2/2022' when run in San Francisco

new Date('2022-03-03').toLocaleDateString('en-US', { timeZone: 'UTC' });
// outputs '3/3/2022' as expected. UTC is used both for declaring and formatting.

new Date('2022-03-03').toLocaleDateString(
  'en-SA-u-ca-islamic-umalqura',
  { timeZone: 'UTC', month: 'long', day: 'numeric', year: 'numeric' }
);
// outputs 'Rajab 30, 1443 AH' as expected

Note that I'm using Date.toLocaleDateString above, but the result is the same as if you'd used Intl.DateTimeFormat.format. The parameters and implementations are the same for both methods.

2. Using the wrong variation of the Islamic calendar (JS implementations typically offer 5 different Islamic calendars!)

A second and more subtle issue is that multiple variations of islamic calendars are used in JavaScript. The list of supported calendars is here: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Intl/Locale/calendars. Excerpting from that page, here are the supported Islamic calendar variations:

islamic

Islamic calendar

islamic-umalqura

Islamic calendar, Umm al-Qura

islamic-tbla

Islamic calendar, tabular (intercalary years [2,5,7,10,13,16,18,21,24,26,29] - astronomical epoch)

islamic-civil

Islamic calendar, tabular (intercalary years [2,5,7,10,13,16,18,21,24,26,29] - civil epoch)

islamic-rgsa

Islamic calendar, Saudi Arabia sighting

(There's also a deprecated islamicc calendar, but islamic-civil should be used instead.)

The default calendar for the ar-SA locale is the islamic-umalqura calendar, not the islamic calendar. To verify:

new Intl.DateTimeFormat('ar-SA').resolvedOptions();
// {
//   calendar: "islamic-umalqura"
//   day: "numeric"
//   locale: "ar-SA"
//   month: "numeric"
//   numberingSystem: "arab"
//   timeZone: "America/Los_Angeles"
//   year: "numeric"
// }

Different Islamic calendar variations will yield different calendar dates. For example:

calendars = ["islamic", "islamic-umalqura", "islamic-tbla", "islamic-civil", "islamic-rgsa"];
options = { timeZone: 'UTC', month: 'long', day: 'numeric', year: 'numeric' };
date = new Date('2022-03-03');
calendars.forEach(calendar => {
  formatted = date.toLocaleDateString(`en-SA-u-ca-${calendar}`, options);
  console.log(`${calendar}: ${formatted}`);
});
// The code above outputs the following:
//   islamic: Shaʻban 1, 1443 AH
//   islamic-umalqura: Rajab 30, 1443 AH
//   islamic-tbla: Rajab 30, 1443 AH
//   islamic-civil: Rajab 29, 1443 AH
//   islamic-rgsa: Shaʻban 1, 1443 AH

3. Bugs in the ICU library used for JS's calendar calculations

A third possible reason for unexpected dates is if there's a bug in the calendar calculation code inside the JS engine. As far as I know, all major browsers delegate their calendar calculations to a library called ICU. If you're using the correct time zone and calendar variation and there's still a problem with the calculation, then you may want to try filing an issue in the ICU JIRA site: https://unicode-org.atlassian.net/jira/software/c/projects/ICU/issues/.

BTW, while answering this question I noticed a bug in the MDN documentation for the Intl.DateTimeFormat constructor where the list of supported calendars is wrong. I filed https://github.com/mdn/content/pull/12764 to fix the content. This PR has already been merged, but it may take a while for the production MDN site to be updated with the fixed content.

Source https://stackoverflow.com/questions/70996839

QUESTION

Javascript: frame precise video stop

Asked 2022-Jan-28 at 14:55

I would like to be able to robustly stop a video when the video arrives on some specified frames in order to do oral presentations based on videos made with Blender, Manim...

I'm aware of this question, but the problem is that the video does not stops exactly at the good frame. Sometimes it continues forward for one frame and when I force it to come back to the initial frame we see the video going backward, which is weird. Even worse, if the next frame is completely different (different background...) this will be very visible.

To illustrate my issues, I created a demo project here (just click "next" and see that when the video stops, sometimes it goes backward). The full code is here.

The important part of the code I'm using is:

      var video = VideoFrame({
          id: 'video',
          frameRate: 24,
          callback: function(curr_frame) {
              // Stops the video when arriving on a frames to stop at.
              if (stopFrames.includes(curr_frame)) {
                  console.log("Automatic stop: found stop frame.");
                  pauseMyVideo();
                  // Ensure we are on the proper frame.
                  video.seekTo({frame: curr_frame});
              }
          }
      });

So far, I avoid this issue by stopping one frame before the end and then using seekTo (not sure how sound this is), as demonstrated here. But as you can see, sometimes when going on the next frame it "freezes" a bit: I guess this is when the stop happens right before the seekTo.

PS: if you know a reliable way in JS to know the number of frames of a given video, I'm also interested.

Concerning the idea to cut the video before hand on the desktop, this could be used... but I had bad experience with that in the past, notably as changing videos sometimes produce some glitches. Also, it can be more complicated to use at it means that the video should be manually cut a lot of time, re-encoded...

EDIT Is there any solution for instance based on WebAssembly (more compatible with old browsers) or Webcodec (more efficient, but not yet wide-spread)? Webcodec seems to allow pretty amazing things, but I'm not sure how to use them for that. I would love to hear solution based on both of them since firefox does not handle webcodec yet. Note that it would be great if audio is not lost in the process. Bonus if I can also make controls appear on request.

EDIT: I'm not sure to understand what's happening here (source)... But it seems to do something close to my need (using webassembly I think) since it manages to play a video in a canvas, with frame... Here is another website that does something close to my need using Webcodec. But I'm not sure how to reliably synchronize sound and video with webcodec.

EDIT: answer to the first question

Concerning the video frame, indeed I chose poorly my frame rate, it was 25 not 24. But even by using a framerate of 25, I still don't get a frame-precise stop, on both Firefox and Chromium. For instance, here is a recording (using OBS) of your demo (I see the same with mine when I use 25 instead of 24):

one frame later, see that the butter "fly backward"(this is maybe not very visible with still screenshots, but see for instance the position of the lower left wing in the flowers):

I can see three potential reasons: first (I think it is the most likely reason), I heard that video.currentTime was not always reporting accurately the time, maybe it could explain why here it fails? It seems to be pretty accurate in order to change the current frame (I can go forward and backward by one frame quite reliably as far as I can see), but people reported here that video.currentTime is computed using the audio time and not the video time in Chromium, leading to some inconsistencies (I observe similar inconsistencies in Firefox), and here that it may either lead the time at which the frame is sent to the compositor or at which the frame is actually printed in the compositor (if it is the latest, it could explain the delay we have sometimes). This would also explain why requestAnimationVideoFrame is better, as it also provides the current media time.

The second reason that could explain that problem is that setInterval may not be precise enough... However requestAnimationFrame is not really better (requestAnimationVideoFrame is not available in Firefox) while it should fire 60 times per seconds which should be quick enough.

The third option I can see is that maybe the .pause function is quite long to fire... and that by the end of the call the video also plays another frame. On the other hand, your example using requestAnimationVideoFrame https://mvyom.csb.app/requestFrame.html seems to work pretty reliably, and it's using .pause! Unfortunately it only works in Chromium, but not in firefox. I see that you use metadata.mediaTime instead of currentTime, maybe this is more precise than current time.

The last option is that there is maybe something subtle concerning vsync as explained in this page. It also reports that expectedDisplayTime may help to solve this issue when using requestAnimationVideoFrame.

ANSWER

Answered 2022-Jan-21 at 19:18

The video has frame rate of 25fps, and not 24fps:

After putting the correct value it works ok: demo
The VideoFrame api heavily relies on FPS provided by you. You can find FPS of your videos offline and send as metadata along with stop frames from server.

The site videoplayer.handmadeproductions.de uses window.requestAnimationFrame() to get the callback.

There is a new better alternative to requestAnimationFrame. The requestVideoFrameCallback(), allows us to do per-video-frame operations on video.
The same functionality, you domed in OP, can be achieved like this:

   const callback = (now, metadata) => {
      if (startTime == 0) {
        startTime = now;
      }
      elapsed = metadata.mediaTime;
      currentFrame = metadata.presentedFrames - doneCount;

      fps = (currentFrame / elapsed).toFixed(3);
      fps = !isFinite(fps) ? 0 : fps;

      updateStats();
      if (stopFrames.includes(currentFrame)) {
        pauseMyVideo();
      } else {
        video.requestVideoFrameCallback(callback);
      }
   };
   video.requestVideoFrameCallback(callback);

And here is how demo looks like.
The API works on chromium based browsers like Chrome, Edge, Brave etc.

There is a JS library, which finds frame rate from video binary file, named mediainfo.js.

Source https://stackoverflow.com/questions/70613008

QUESTION

In the ompr package in R, how can I rephrase my objective/constraints/variables so as to avoid the "problem too large" error?

Asked 2022-Jan-21 at 19:43

I am trying to learn to fit a linear integer programming optimization model in R using the ompr package that a colleague had previously fit using CPLEX/GAMS (specifically, the one described here: Haight et al. 2021). I am running my implementation on a Linux Supercomputing server at my University that has 248gb of memory, which I'd think would be sufficient for the job.

Here is my code and output from the failure report from the server:

#Read in the necessary pre-generated data and packages

library(pacman); library(dplyr); library(ROI); library(ompr); library(ompr.roi)
n.ij = readRDS(file="nij1.rds") #An indexing vector.
B = 10 #Budget constraint--inspect only 10 lakes maximum

#Initialize model prior to setting the objective.
mod1 = MILPModel() %>% 
add_variable(u[i, j], type = "binary", i = 1:n.ij, j = 1:n.ij) %>%
add_variable(x[i], type = "binary", i = 1:n.ij) %>% 
add_variable(x[j], type = "binary", j = 1:n.ij) %>% 
add_constraint(x[i] + x[j] >= u[i,j], i = 1:n.ij, j = 1:n.ij) %>% 
add_constraint(sum_expr(x[i], i = 1:n.ij) <= B)
 
#Read in the relevant adjacency matrix of boat movements between every pair of lakes.
boats.n.ij = readRDS(file="boatsnij1.rds")

#Some system and object size info.
system(paste0("cat /proc/",Sys.getpid(),"/status | grep VmSize"))
VmSize: 13017708 kB
object.size(mod1)
6798778288 bytes

#Now, set objective with this specific boats.n.ij file.
mod1.full = mod1 %>% 
set_objective(sum_expr(u[i,j] * boats.n.ij[i, j], i = 1:n.ij, j = 1:n.ij))

Error in subCsp_ij(x, i, j, drop = drop) : 
  Cholmod error 'problem too large' at file ../Core/cholmod_sparse.c, line 89
Calls: %>% ... [ -> callGeneric -> eval -> eval -> [ -> [ -> subCsp_ij
Execution halted

For the purposes of creating a reproducible example, mock versions of n.ij and boats.n.ij can be generated as follows:

library(Matrix)

boats = rpois(7940*7940, 2)
keep = sample(c(0,1), 7940*7940, replace=T, prob = c(0.8, 0.2))
boat.dat = boats*keep

boats.n.ij = matrix(boat.dat, nrow=7940, ncol=7940)
diag(boats.n.ij) = 0
boats.n.ij = Matrix(boats.n.ij, sparse = T)

boats.n.ij[1:10, 1:10]

n.ij = 1:7940

Why am I failing to add the objective to my model? Is it just that I am implying the existence of three very large matrices (the decision matrix u, the boats.n.ij matrix, and their product matrix)? Is it because the model is already a file that is about 6.8gb? Is there a cap on memory or object size imposed by R I am running into? Are these functions just not capable of considering an objective with this many decision points?

I can confirm that I have been able to run a scaled-down version of the model on a very small subset of boats.n.ij that optimizes just fine, so I don't think it's an issue with my model specification, but I could be wrong...I should also state explicitly I am not interested in solutions that do not involve solving this model in R, as that is the express objective here. I am open, however, to using other packages if there's a more robust one available (although I like this one otherwise).

Note: Unlike in the paper I've cited, I have eliminated the need for a vector called b.ij that my colleague does use, so that isn't the issue here.

Edit: Note that @nicola's reforming of the objective will set and solve, but the original constraints and/or variables would no longer have the same relationship with it, so it'd be fitting a different model than the one I want to fit. In the original construction, only a max of 10 values in x[i], and thus a max of 10 unique values of i within the decision variable u[i,j], would be allowed to be 1s thanks to the constraint involving our budget parameter B. In @nicola's version, far more than 10 unique values of i are permitted to be 1s within u[i,j]. It's actually unclear to me at least how the constraints as originally written interact with @nicola's objective, if at all. However, I suspect an objective like @nicola's could definitely be used to exploit the sparsity of my boats.n.ij matrix so as to avoid the "problem too large" error, but it would require the variables and/constraints to be modified accordingly. I changed the title of the question to be much clearer as to what I am looking for--I want to avoid the error but otherwise fit an equivalent model.

Second edit: @nicola's solution does work after all! However, the variables and constraints needed a little modification due to updates to ompr since I posted this question. See the following toy example:

library(Matrix)
library(slam)
library(dplyr)
library(tidyr)
library(ROI)
library(ompr)
library(ompr.roi)
library(Rglpk)
library(ROI.plugin.glpk)
library(lattice)

set.seed(101)
N = 500
boats = rpois(N*N, 2)
keep = sample(c(0,1), N*N, replace=T, prob = c(0.97, 0.03))
boat.dat = boats*keep

boats.n.ij = Matrix(boat.dat, nrow=N, ncol=N, sparse =T)
diag(boats.n.ij) = 0

boats.n.ij[1:10, 1:10]

n.ij = N
B = 5

mod1 = MIPModel() %>% 
  add_variable(u[i, j], type = "binary", i = 1:n.ij, j = 1:n.ij) %>%
  add_variable(x[i], type = "binary", i = 1:n.ij) %>% 
  add_variable(y[j], type = "binary", j = 1:n.ij) %>% 
  add_constraint(x[i] == y[j], i = 1:n.ij, j = 1:n.ij, i == j) %>% 
  add_constraint(sum_over(x[i], i = 1:n.ij) <= B) %>% 
  add_constraint(u[i,j] <= x[i] + y[j], i = 1:n.ij, j = 1:n.ij)

boatsSTM = as.simple_triplet_matrix(boats.n.ij)

#setting the objective function
mod.2nd = mod1 %>% set_objective(sum_over(u[boatsSTM$i[k], boatsSTM$j[k]] * boatsSTM$v[k], k = 1:length(boatsSTM$i)))

mod.2nd.solved = mod.2nd %>% 
  solve_model(with_ROI("glpk", verbose=TRUE))


testB = get_solution(mod.2nd.solved, u[i,j])
test2B = pivot_wider(testB, names_from = j, values_from = value) %>% dplyr::select(-variable, -i)
test3B = as.matrix(test2B, nrow=100)

levelplot(test3B)

ANSWER

Answered 2021-Dec-20 at 15:28

An attempt:

require(slam)
boatsSTM<-as.simple_triplet_matrix(boats.n.ij)
...

#setting the objective function
set_objective(sum_expr(u[boatsSTM$i[k], boatsSTM$j[k]] * boatsSTM$v[k], k = 1:length(boatsSTM$i)))

We exploit the sparsity of the matrix. In a simple triplet matrix you just list the values that are not zero, implying that if an element is not listed is equal to zero. The values are denoted with a (i, j, v) triplet, where i denotes the row index, j the column index and v the value. So, for instance, a (2, 4, 10.32) triplet denotes that m[2, 4] = 10.32.

In your sum_expr line, we exploit this and just add the elements that are different from zero. We don't multiply each element of u with each element of boats, since most are zero and irrelevant to the sum; rather we just do the above for the elements that matter.

The slam package implements the simple triplet matrix which is, at its root, just a list of i, j and v values.

Source https://stackoverflow.com/questions/70276554

QUESTION

A clarification on the named requirements for containers

Asked 2022-Jan-01 at 16:27

I am trying to get to grips with the specifics of the (C++20) standards requirements for container classes with a view to writing some container classes that are compatible with the standard library. To begin looking into this matter I have looked up the references for named requirements, specifically around container requirements, and have only found one general container requirement called Container given by the standard. Reading this requirement has given my two queries that I am unsure about and would like some clarification on:

The requirement for the expression a == b for two container type C has as precondition on the element type T that it is equality comparable. However, noted later on the same page under the header 'other requirements' is the explicitly requirement that T be always equality comparable. Thus, on my reading the precondition for the aforementioned requirement is redundant and need not be given. Am I correct in this thinking, or is there something else at play here that I should take into account?
I was surprised to see explicit requirements on T at all: notably the equality comparable requirement above and the named requirement destructible. Does this mean it is undefined behaviour to ever construct standard containers of types failing these requirements, or only to perform certain standard library function calls on them?

Apologies if these two questions sound asinine, I am currently trying to transition my C++ knowledge from a place of having a basic understanding of how to use features to a robust understanding so that I may write good generic code. Whilst I am trying to use (a draft of) the standard to look up behaviour where possible, its verbiage is oft too verbose for me to completely understand what is actually being said.

In an attempt to seek the answer I cooked up a a quick test .cpp file to try an compile, given below. All uncommented code compiles with MSVC compiler set to C++20. All commented code will not compile, and visa versa all uncommented code will. It seems that what one naively thinks should work does In particular:

We cannot construct any object without a destructor, though the objects type is valid and can be used for other things (for example as a template parameter!)
We cannot create an object of vector<T>, where T has no destructor, even if we don't attempt to create any objects T. Presumably because creating the destructor for vector<T> tries to access a destructor for T.
We can create an object of type vector<T>, T where T has no operator ==, so long as we do not try to use operator ==, which would require T to have operator ==.

However, just because my compiler lets me make an object of vector<T> where T is not equality-comparable does not mean I have achieved standards compliant behaviour/ all of our behaviour is not undefined - which is what I want I concerned about, especially as at least some of the usual requirements on the container object have been violated.

Code:

#include<vector>
#include<iostream>

//A basic class with its == operator removed
class MyNonEqualityComparableClass
{
    //Destroy any possible == operator, for good measure
    template<typename T>
    auto operator==(T&& t) = delete;

    public:
        //Give the test struct a value so we mute the class and check
        bool Mytestvalue = true;
};

class MyNonDestructableClass
{
    ~MyNonDestructableClass() = delete;
};



// A basic class template with no functionality added
template<typename T>
class MyTemplateClass 
{};

int main()
{


//1. Non-destructable Class Test.
{
    
    //Compiler Grumbles if uncommented, no destructor is inaccessable for the class
    //MyNonDestructableClass a, b;

    //Class Dependent On my Nondestructable Class
    MyTemplateClass < MyNonDestructableClass> mtc;

    //Compiler Grumbles if uncommented, as class tries to acceess the inacessable destructor of MyNonDestructableClass
    //auto u = std::vector< MyNonDestructableClass>();

    
}

//2. Non-Equality-Comparable Class Test.
{
    
    MyNonEqualityComparableClass x, y;
    //Compiler Grumbles if uncommented 
    //x == y;

    //Compiler Fine with below
    auto u = std::vector<MyNonEqualityComparableClass>();
    auto v = std::vector<MyNonEqualityComparableClass>();
    u.push_back(x);
    v.push_back(y);
    x.Mytestvalue = !x.Mytestvalue;

    //Compiler Grumbles if uncommented 
    //u == v;

    std::cout << "The value of x.Mytestvalue is: \t" << std::boolalpha << x.Mytestvalue << std::endl;
}

}

My output:

The value of x.Mytestvalue is:  false

ANSWER

Answered 2021-Dec-30 at 04:32

If the members of a container are not destructible, then the container could never do anything except add new elements (or replace existing elements). erase, resize and destruction all involve destroying elements. If you had a type T that was not destructible, and attempted to instantiate a vector<T> (say), I would expect that it would fail to compile.

As for the duplicate requirements, I suspect that's just something that snuck in when the CppReference folks wrote that page. The container requirements in the standard mention (in the entry for a == b) that the elements must be equality comparable.

Source https://stackoverflow.com/questions/70527058

QUESTION

Why does summary() show different standard errors than coeftest()?

Asked 2021-Dec-26 at 14:06

I run a glm() using robust standard errors. For a subsequent model comparison I calculate the difference of two regression models (coefficients & se). For that calculation I use the summary() function. However, the summary function of the models show different standard errors than the ones I get from coeftest(). Values for coefficients remain identical.

Input:

mod.01 <- glm(dep ~ indep1 + indep2 + indep3,
          family = binomial (link = "logit"), data = data)
coeftest(mod.01, vcov. = vcovHC, type= "HC3", df = NULL)

summary(mod.01, robust=T)

Output:

coeftest()      
                           Estimate  Std. Error  t value  Pr(>|t|)    
    (Intercept)         -2.72917626  0.16367787 -16.6741 < 2.2e-16 ***
    indep1               0.00427870  0.41928906   0.0102  0.991859    
    indep2               2.00243724  0.19757861  10.1349 < 2.2e-16 ***
    indep3               0.36385098  0.32783817   1.1098  0.267159    


summary()
                           Estimate Std. Error z value Pr(>|z|)    
    (Intercept)          -2.7291763  0.1758744 -15.518  < 2e-16 ***
    indep1                0.0042787  0.3389472   0.013  0.98993    
    indep2                2.0024372  0.1746829  11.463  < 2e-16 ***
    indep3                0.3638510  0.2604196   1.397  0.16236

How do I get the robust se of mod.01 into the summary function? So that eventually my subsequent calculation includes the correct robust se will also be shown shown in the regression table.

Thanks in advance!

ANSWER

Answered 2021-Dec-26 at 11:33

The merits of lmtest::coeftest is that it is possible to use a different covariance matrix than computed by lm().

fit <- glm(am ~ hp + cyl, family=binomial(link="logit"), mtcars)

summary(fit)
#             Estimate Std. Error z value Pr(>|z|)   
# (Intercept)  5.83220    2.06595   2.823  0.00476 **
# hp           0.02775    0.01366   2.031  0.04228 * 
# cyl         -1.70306    0.60286  -2.825  0.00473 **

If you put the initial covariance matrix from lm() into coeftest, the standard errors are identical.

library(sandwich); library(lmtest)

coeftest(fit, vcov.=vcov(fit))
#              Estimate Std. Error z value Pr(>|z|)   
# (Intercept)  5.832204   2.065949  2.8230 0.004757 **
# hp           0.027748   0.013664  2.0307 0.042282 * 
# cyl         -1.703064   0.602862 -2.8250 0.004729 **

Compute robust standard errors

Now, using coeftest we may specify a different covariance matrix. (Sidenote: The actual one with "robust" standard errors, also referred to as white standard errors, is specified with type='HC0', the other 'HC*' are refinements, e.g. Stata uses 'HC1').

(ct <- coeftest(fit, vcov.=vcovHC(fit, type='HC0')))
#              Estimate Std. Error z value Pr(>|z|)   
# (Intercept)  5.832204   2.139307  2.7262 0.006407 **
# hp           0.027748   0.012254  2.2643 0.023553 * 
# cyl         -1.703064   0.572045 -2.9772 0.002909 **

That's how you get different standard errors from coeftest.

Publishing table with robust standard errors

To put those corrected standard errors in a table, here's a way using texreg::screenreg. You may also us texreg::texreg for LaTeX or texreg::htmlreg for HTML, body stays basically the same.

Here with conventional standard errors,

texreg::screenreg(fit)
# =========================
#                 Model 1  
# -------------------------
# (Intercept)       5.83 **
#                  (2.07)  
# hp                0.03 * 
#                  (0.01)  
# cyl              -1.70 **
#                  (0.60)  
# -------------------------
# AIC              34.63   
# BIC              39.02   
# Log Likelihood  -14.31   
# Deviance         28.63   
# Num. obs.        32      
# =========================
# *** p < 0.001; ** p < 0.01; * p < 0.05

and here with robust standard errors, both, p-values and standard errors need to be overrided.

texreg::screenreg(fit, 
                  override.pvalues=ct[, 4],
                  override.se=ct[, 3])
# =========================
#                 Model 1  
# -------------------------
# (Intercept)       5.83 **
#                  (2.73)  
# hp                0.03 * 
#                  (2.26)  
# cyl              -1.70 **
#                 (-2.98)  
# -------------------------
# AIC              34.63   
# BIC              39.02   
# Log Likelihood  -14.31   
# Deviance         28.63   
# Num. obs.        32      
# =========================
# *** p < 0.001; ** p < 0.01; * p < 0.05

Source https://stackoverflow.com/questions/70485809

QUESTION

Is mass assignment protection needed when properly using validation?

Asked 2021-Nov-20 at 14:46

I've been using Laravel (5.4) and PHP both for the very first time in a legacy app I've been tasked to maintain, and over the past couple of months I have noticed a pain point, which is mass assignment.

I believe I have a well-enough grasp of what is is, but I am not sure if I am really utilizing it correctly. The official documentation illustrates the problem using a (good, but extreme) example of a user's role. I understand the significance of this and why I should guard such variables. However, I don't think I really have a solid understanding of determining what fields to guard or not (if at all), in general.

It seems that a lot of the nice features of Laravel don't seem to have mass-assignment in mind, which is the main source of my frustration. For example, if I had an endpoint that had some optional fields (i.e. you could just not specify them in the request), then doing a mymodel->update(request->all()) would only update the fields that you provided. If many of the fields were guarded, I would have to have many repeated isset() checks if I wanted to achieve the same behaviour, which seems unnecessary. I also know that request->all() shouldn't really be used like this, but I am just trying to illustrate a point.

It becomes more cumbersome when you consider that you're probably going to use robust validation alongside something like request->only([...]) to make sure your data is correct and filtered to only what you expect.

So ultimately what I am asking is:

In the presence of robust validation and input-shaping methods, is mass-assigning nearly anything still worth doing? In general it seems that I am jumping through so many more hoops for no reason, when it seems that my validation steps already took care of the problem. If it is still worth doing, what am I missing? Should mass-assignment protection only be delegated to super-important fields, like a user's role, and nothing else?

ANSWER

Answered 2021-Nov-20 at 14:46

Actually i didnt understand your mean by this sentence:

If many of the fields were guarded, I would have to have many repeated isset() checks if I wanted to achieve the same behavior

Not Mass-Assignable fields, just get discarded when presented in the request

But in the scenario when user pass a parameter like forbidden_param from the request and forbidden_param is not mass assignable ( either is not in fillables or it is in the guarded) there would be no error thrown, and the eloquent is going to discard the forbidden_param .

if you had this in your code:

if(isset($request->forbidden_param))

you can get relieve from them, when using mass assignable.

Laravel Lets you to do a job any how you want

Laravel just doesn't force you to use helpers or Facade or DI , ... to use a service.

for protecting the Models fields you can come with different solutions to:

1.Eloquent protection (Mass Assignable)

you can pass the parameters from request, then protect them from the eloquent.

//in controller
MyModel::update($request->input())

// in model

$fillable = [ ... ]
//or
$guarded = [ ... ]

it is a common pattern & I have seen lot of codes using this scenario.

The benefits are:

leaning the controller
not worry about fallible fieleds, if you just forgot to control input in other places (controller / validation)

2.Controller Protection

another pattern which I have seen is placing the exact parameters in the update method, in the controller.

Model::update([
    'permitted_param1' => $request->permitted_param
    'permitted_param2' => $request->permitted_param2
]);

in this case one would relax the mass assignable in the model with:

$guarded = [];

or can benefit it, in case of if another teammate on another controller may would forgotten to control the inputs & use $request->input(). this would be optional though.

benefits of doing this:

Ability to have different input names and database field names
Ability to manipulate an input before put into model (which can be achieved using mutators or repository ). but in case of few fields to change it is simpler to implement.

in example below, controller use a mixed approach:

User::create($request->except('password') + [ 'secret' => bcrypt($request->password)] );

3.Protecting in Validator

One of the most promising methods, I have seen for this, is to use only validated data, like below: for Laravel 8+:

MyModel::update($request->safe()->all());

and for Laravel before 8:

MyModel::update($request->validated())

and you can either relax the mass assignable

Benefits:

you have combined the validating and sensitizing data in one place
leaner controller
you are sure that you have validated all your data

Note: in case of optional data you can just not to put the required rule

4.Repository Pattern

another approach for this, is to use the Repository Pattern which can introduce complexity for small projects.

but results in :

leaner controllers & models
move query logic in repository
make queries more reusable and decoupled

for projects using this design pattern, this control can be move to repository.

Source https://stackoverflow.com/questions/70044107

QUESTION

How can I specialize std::common_type<A,B> so that it's naturally commutative?

Asked 2021-Nov-16 at 09:56

std::common_type<T1, ..., TN> is a helper template in C++ which can find the common type which all of T1 ... TN are implicitly convertible to.

According the C++ spec, a user may specialize std::common_type<T1,T2> if certain conditions apply, and:

std::common_type<T1, T2>::type and std::common_type<T2, T1>::type must denote the same type.

However, common_type<T1, T2> might be a very complicated specialization for user types T1 and T2:

namespace std {

    template <typename T1, complicated_constraint_of<T1> T2, ...>
    struct common_type<complicated_expression_of<T1, ...>, complicated_expression_of<T2, ...>> {
        using type = complicated_type_expression_of<T1,T2>;
    };

}

In general, the constraint expressions are not necessarily symmetrical (for example, we might specify that T2 is a base of T1). This means that to preserve symmetry, we'd need to rewrite the entire specialization with T1 and T2 reversed, but doing that without making any mistake is extremely difficult and fragile.

How can I robustly define a commutative specialization of common_type<T1,T2> for my own types?

ANSWER

Answered 2021-Nov-16 at 08:54

Here is the C++20 solution I came up with:

// define concept of `common_type<A,B>` existing
template <typename A, typename B>
concept has_in_common_ordered = requires { common_type<A,B>::type; };

namespace std {
    
    // define common_type<A,B> if common_type<B,A>::type exists:
    template <typename A, has_in_common_ordered<A> B>
    struct common_type<A,B> : public common_type<B,A> {};
    
}

Now, with the above, the following should compile:

struct X {};
struct Y {};

namespace std {
    
    template<>
    struct common_type<X,Y> {
        using type = X; // or whatever
    };
    
}

int main() {
    // even though we only specialized for one ordering,
    // both orderings now exist:
    std::common_type<X,Y>::type a;
    std::common_type<Y,X>::type b;
}

I think one of two things is possible:

This is a natural technique that should really just be part of the standard definition of std::common_type.
This is extremely evil and you should never do it, for subtle reasons.

I await someone to tell me which one it is in the comments. ;)

Source https://stackoverflow.com/questions/69984704

Community Discussions, Code Snippets contain sources that include Stack Exchange Network

Vulnerabilities

No vulnerabilities reported

Install Robust

You can download it from GitHub.
You can use Robust like any standard Java library. Please include the the jar files in your classpath. You can also use any IDE and you can run and debug the Robust component as you would do with any other Java program. Best practice is to use a build tool that supports dependency management such as Maven or Gradle. For Maven installation, please refer maven.apache.org. For Gradle installation, please refer gradle.org .

Support

For any new features, suggestions and bugs create an issue on GitHub. If you have any questions check and ask questions on community page Stack Overflow .