kandi background
Explore Kits

clean-code-javascript | : bathtub : Clean Code concepts | Architecture library

 by   ryanmcdermott JavaScript Version: Current License: MIT

 by   ryanmcdermott JavaScript Version: Current License: MIT

Download this library from

kandi X-RAY | clean-code-javascript Summary

clean-code-javascript is a JavaScript library typically used in Architecture applications. clean-code-javascript has no bugs, it has no vulnerabilities, it has a Permissive License and it has medium support. You can download it from GitHub.
Software engineering principles, from Robert C. Martin's book Clean Code, adapted for JavaScript. This is not a style guide. It's a guide to producing readable, reusable, and refactorable software in JavaScript. Not every principle herein has to be strictly followed, and even fewer will be universally agreed upon. These are guidelines and nothing more, but they are ones codified over many years of collective experience by the authors of Clean Code. Our craft of software engineering is just a bit over 50 years old, and we are still learning a lot. When software architecture is as old as architecture itself, maybe then we will have harder rules to follow. For now, let these guidelines serve as a touchstone by which to assess the quality of the JavaScript code that you and your team produce.
Support
Support
Quality
Quality
Security
Security
License
License
Reuse
Reuse

kandi-support Support

  • clean-code-javascript has a medium active ecosystem.
  • It has 56019 star(s) with 7108 fork(s). There are 1658 watchers for this library.
  • It had no major release in the last 12 months.
  • There are 34 open issues and 72 have been closed. On average issues are closed in 146 days. There are 8 open pull requests and 0 closed requests.
  • It has a neutral sentiment in the developer community.
  • The latest version of clean-code-javascript is current.
This Library - Support
Best in #Architecture
Average in #Architecture
This Library - Support
Best in #Architecture
Average in #Architecture

quality kandi Quality

  • clean-code-javascript has 0 bugs and 0 code smells.
This Library - Quality
Best in #Architecture
Average in #Architecture
This Library - Quality
Best in #Architecture
Average in #Architecture

securitySecurity

  • clean-code-javascript has no vulnerabilities reported, and its dependent libraries have no vulnerabilities reported.
  • clean-code-javascript code analysis shows 0 unresolved vulnerabilities.
  • There are 0 security hotspots that need review.
This Library - Security
Best in #Architecture
Average in #Architecture
This Library - Security
Best in #Architecture
Average in #Architecture

license License

  • clean-code-javascript is licensed under the MIT License. This license is Permissive.
  • Permissive licenses have the least restrictions, and you can use them in most projects.
This Library - License
Best in #Architecture
Average in #Architecture
This Library - License
Best in #Architecture
Average in #Architecture

buildReuse

  • clean-code-javascript releases are not available. You will need to build from source code and install.
  • Installation instructions are not available. Examples and code snippets are available.
This Library - Reuse
Best in #Architecture
Average in #Architecture
This Library - Reuse
Best in #Architecture
Average in #Architecture
Top functions reviewed by kandi - BETA

Coming Soon for all Libraries!

Currently covering the most popular Java, JavaScript and Python libraries. See a SAMPLE HERE.
kandi's functional review helps you automatically verify the functionalities of the libraries and avoid rework.

clean-code-javascript Key Features

:bathtub: Clean Code concepts adapted for JavaScript

Use meaningful and pronounceable variable names

copy iconCopydownload iconDownload
const yyyymmdstr = moment().format("YYYY/MM/DD");

Use the same vocabulary for the same type of variable

copy iconCopydownload iconDownload
getUserInfo();
getClientData();
getCustomerRecord();

Use searchable names

copy iconCopydownload iconDownload
// What the heck is 86400000 for?
setTimeout(blastOff, 86400000);

Use explanatory variables

copy iconCopydownload iconDownload
const address = "One Infinite Loop, Cupertino 95014";
const cityZipCodeRegex = /^[^,\\]+[,\\\s]+(.+?)\s*(\d{5})?$/;
saveCityZipCode(
  address.match(cityZipCodeRegex)[1],
  address.match(cityZipCodeRegex)[2]
);

Avoid Mental Mapping

copy iconCopydownload iconDownload
const locations = ["Austin", "New York", "San Francisco"];
locations.forEach(l => {
  doStuff();
  doSomeOtherStuff();
  // ...
  // ...
  // ...
  // Wait, what is `l` for again?
  dispatch(l);
});

Don't add unneeded context

copy iconCopydownload iconDownload
const Car = {
  carMake: "Honda",
  carModel: "Accord",
  carColor: "Blue"
};

function paintCar(car, color) {
  car.carColor = color;
}

Use default arguments instead of short circuiting or conditionals

copy iconCopydownload iconDownload
function createMicrobrewery(name) {
  const breweryName = name || "Hipster Brew Co.";
  // ...
}

Function arguments (2 or fewer ideally)

copy iconCopydownload iconDownload
function createMenu(title, body, buttonText, cancellable) {
  // ...
}

createMenu("Foo", "Bar", "Baz", true);

Functions should do one thing

copy iconCopydownload iconDownload
function emailClients(clients) {
  clients.forEach(client => {
    const clientRecord = database.lookup(client);
    if (clientRecord.isActive()) {
      email(client);
    }
  });
}

Function names should say what they do

copy iconCopydownload iconDownload
function addToDate(date, month) {
  // ...
}

const date = new Date();

// It's hard to tell from the function name what is added
addToDate(date, 1);

Functions should only be one level of abstraction

copy iconCopydownload iconDownload
function parseBetterJSAlternative(code) {
  const REGEXES = [
    // ...
  ];

  const statements = code.split(" ");
  const tokens = [];
  REGEXES.forEach(REGEX => {
    statements.forEach(statement => {
      // ...
    });
  });

  const ast = [];
  tokens.forEach(token => {
    // lex...
  });

  ast.forEach(node => {
    // parse...
  });
}

Remove duplicate code

copy iconCopydownload iconDownload
function showDeveloperList(developers) {
  developers.forEach(developer => {
    const expectedSalary = developer.calculateExpectedSalary();
    const experience = developer.getExperience();
    const githubLink = developer.getGithubLink();
    const data = {
      expectedSalary,
      experience,
      githubLink
    };

    render(data);
  });
}

function showManagerList(managers) {
  managers.forEach(manager => {
    const expectedSalary = manager.calculateExpectedSalary();
    const experience = manager.getExperience();
    const portfolio = manager.getMBAProjects();
    const data = {
      expectedSalary,
      experience,
      portfolio
    };

    render(data);
  });
}

Set default objects with Object.assign

copy iconCopydownload iconDownload
const menuConfig = {
  title: null,
  body: "Bar",
  buttonText: null,
  cancellable: true
};

function createMenu(config) {
  config.title = config.title || "Foo";
  config.body = config.body || "Bar";
  config.buttonText = config.buttonText || "Baz";
  config.cancellable =
    config.cancellable !== undefined ? config.cancellable : true;
}

createMenu(menuConfig);

Don't use flags as function parameters

copy iconCopydownload iconDownload
function createFile(name, temp) {
  if (temp) {
    fs.create(`./temp/${name}`);
  } else {
    fs.create(name);
  }
}

Avoid Side Effects (part 1)

copy iconCopydownload iconDownload
// Global variable referenced by following function.
// If we had another function that used this name, now it'd be an array and it could break it.
let name = "Ryan McDermott";

function splitIntoFirstAndLastName() {
  name = name.split(" ");
}

splitIntoFirstAndLastName();

console.log(name); // ['Ryan', 'McDermott'];

Avoid Side Effects (part 2)

copy iconCopydownload iconDownload
const addItemToCart = (cart, item) => {
  cart.push({ item, date: Date.now() });
};

Don't write to global functions

copy iconCopydownload iconDownload
Array.prototype.diff = function diff(comparisonArray) {
  const hash = new Set(comparisonArray);
  return this.filter(elem => !hash.has(elem));
};

Favor functional programming over imperative programming

copy iconCopydownload iconDownload
const programmerOutput = [
  {
    name: "Uncle Bobby",
    linesOfCode: 500
  },
  {
    name: "Suzie Q",
    linesOfCode: 1500
  },
  {
    name: "Jimmy Gosling",
    linesOfCode: 150
  },
  {
    name: "Gracie Hopper",
    linesOfCode: 1000
  }
];

let totalOutput = 0;

for (let i = 0; i < programmerOutput.length; i++) {
  totalOutput += programmerOutput[i].linesOfCode;
}

Encapsulate conditionals

copy iconCopydownload iconDownload
if (fsm.state === "fetching" && isEmpty(listNode)) {
  // ...
}

Avoid negative conditionals

copy iconCopydownload iconDownload
function isDOMNodeNotPresent(node) {
  // ...
}

if (!isDOMNodeNotPresent(node)) {
  // ...
}

Avoid conditionals

copy iconCopydownload iconDownload
class Airplane {
  // ...
  getCruisingAltitude() {
    switch (this.type) {
      case "777":
        return this.getMaxAltitude() - this.getPassengerCount();
      case "Air Force One":
        return this.getMaxAltitude();
      case "Cessna":
        return this.getMaxAltitude() - this.getFuelExpenditure();
    }
  }
}

Avoid type-checking (part 1)

copy iconCopydownload iconDownload
function travelToTexas(vehicle) {
  if (vehicle instanceof Bicycle) {
    vehicle.pedal(this.currentLocation, new Location("texas"));
  } else if (vehicle instanceof Car) {
    vehicle.drive(this.currentLocation, new Location("texas"));
  }
}

Avoid type-checking (part 2)

copy iconCopydownload iconDownload
function combine(val1, val2) {
  if (
    (typeof val1 === "number" && typeof val2 === "number") ||
    (typeof val1 === "string" && typeof val2 === "string")
  ) {
    return val1 + val2;
  }

  throw new Error("Must be of type String or Number");
}

Don't over-optimize

copy iconCopydownload iconDownload
// On old browsers, each iteration with uncached `list.length` would be costly
// because of `list.length` recomputation. In modern browsers, this is optimized.
for (let i = 0, len = list.length; i < len; i++) {
  // ...
}

Remove dead code

copy iconCopydownload iconDownload
function oldRequestModule(url) {
  // ...
}

function newRequestModule(url) {
  // ...
}

const req = newRequestModule;
inventoryTracker("apples", req, "www.inventory-awesome.io");

Use getters and setters

copy iconCopydownload iconDownload
function makeBankAccount() {
  // ...

  return {
    balance: 0
    // ...
  };
}

const account = makeBankAccount();
account.balance = 100;

Make objects have private members

copy iconCopydownload iconDownload
const Employee = function(name) {
  this.name = name;
};

Employee.prototype.getName = function getName() {
  return this.name;
};

const employee = new Employee("John Doe");
console.log(`Employee name: ${employee.getName()}`); // Employee name: John Doe
delete employee.name;
console.log(`Employee name: ${employee.getName()}`); // Employee name: undefined

Prefer ES2015/ES6 classes over ES5 plain functions

copy iconCopydownload iconDownload
const Animal = function(age) {
  if (!(this instanceof Animal)) {
    throw new Error("Instantiate Animal with `new`");
  }

  this.age = age;
};

Animal.prototype.move = function move() {};

const Mammal = function(age, furColor) {
  if (!(this instanceof Mammal)) {
    throw new Error("Instantiate Mammal with `new`");
  }

  Animal.call(this, age);
  this.furColor = furColor;
};

Mammal.prototype = Object.create(Animal.prototype);
Mammal.prototype.constructor = Mammal;
Mammal.prototype.liveBirth = function liveBirth() {};

const Human = function(age, furColor, languageSpoken) {
  if (!(this instanceof Human)) {
    throw new Error("Instantiate Human with `new`");
  }

  Mammal.call(this, age, furColor);
  this.languageSpoken = languageSpoken;
};

Human.prototype = Object.create(Mammal.prototype);
Human.prototype.constructor = Human;
Human.prototype.speak = function speak() {};

Use method chaining

copy iconCopydownload iconDownload
class Car {
  constructor(make, model, color) {
    this.make = make;
    this.model = model;
    this.color = color;
  }

  setMake(make) {
    this.make = make;
  }

  setModel(model) {
    this.model = model;
  }

  setColor(color) {
    this.color = color;
  }

  save() {
    console.log(this.make, this.model, this.color);
  }
}

const car = new Car("Ford", "F-150", "red");
car.setColor("pink");
car.save();

Prefer composition over inheritance

copy iconCopydownload iconDownload
class Employee {
  constructor(name, email) {
    this.name = name;
    this.email = email;
  }

  // ...
}

// Bad because Employees "have" tax data. EmployeeTaxData is not a type of Employee
class EmployeeTaxData extends Employee {
  constructor(ssn, salary) {
    super();
    this.ssn = ssn;
    this.salary = salary;
  }

  // ...
}

Single Responsibility Principle (SRP)

copy iconCopydownload iconDownload
class UserSettings {
  constructor(user) {
    this.user = user;
  }

  changeSettings(settings) {
    if (this.verifyCredentials()) {
      // ...
    }
  }

  verifyCredentials() {
    // ...
  }
}

Open/Closed Principle (OCP)

copy iconCopydownload iconDownload
class AjaxAdapter extends Adapter {
  constructor() {
    super();
    this.name = "ajaxAdapter";
  }
}

class NodeAdapter extends Adapter {
  constructor() {
    super();
    this.name = "nodeAdapter";
  }
}

class HttpRequester {
  constructor(adapter) {
    this.adapter = adapter;
  }

  fetch(url) {
    if (this.adapter.name === "ajaxAdapter") {
      return makeAjaxCall(url).then(response => {
        // transform response and return
      });
    } else if (this.adapter.name === "nodeAdapter") {
      return makeHttpCall(url).then(response => {
        // transform response and return
      });
    }
  }
}

function makeAjaxCall(url) {
  // request and return promise
}

function makeHttpCall(url) {
  // request and return promise
}

Liskov Substitution Principle (LSP)

copy iconCopydownload iconDownload
class Rectangle {
  constructor() {
    this.width = 0;
    this.height = 0;
  }

  setColor(color) {
    // ...
  }

  render(area) {
    // ...
  }

  setWidth(width) {
    this.width = width;
  }

  setHeight(height) {
    this.height = height;
  }

  getArea() {
    return this.width * this.height;
  }
}

class Square extends Rectangle {
  setWidth(width) {
    this.width = width;
    this.height = width;
  }

  setHeight(height) {
    this.width = height;
    this.height = height;
  }
}

function renderLargeRectangles(rectangles) {
  rectangles.forEach(rectangle => {
    rectangle.setWidth(4);
    rectangle.setHeight(5);
    const area = rectangle.getArea(); // BAD: Returns 25 for Square. Should be 20.
    rectangle.render(area);
  });
}

const rectangles = [new Rectangle(), new Rectangle(), new Square()];
renderLargeRectangles(rectangles);

Interface Segregation Principle (ISP)

copy iconCopydownload iconDownload
class DOMTraverser {
  constructor(settings) {
    this.settings = settings;
    this.setup();
  }

  setup() {
    this.rootNode = this.settings.rootNode;
    this.settings.animationModule.setup();
  }

  traverse() {
    // ...
  }
}

const $ = new DOMTraverser({
  rootNode: document.getElementsByTagName("body"),
  animationModule() {} // Most of the time, we won't need to animate when traversing.
  // ...
});

Dependency Inversion Principle (DIP)

copy iconCopydownload iconDownload
class InventoryRequester {
  constructor() {
    this.REQ_METHODS = ["HTTP"];
  }

  requestItem(item) {
    // ...
  }
}

class InventoryTracker {
  constructor(items) {
    this.items = items;

    // BAD: We have created a dependency on a specific request implementation.
    // We should just have requestItems depend on a request method: `request`
    this.requester = new InventoryRequester();
  }

  requestItems() {
    this.items.forEach(item => {
      this.requester.requestItem(item);
    });
  }
}

const inventoryTracker = new InventoryTracker(["apples", "bananas"]);
inventoryTracker.requestItems();

Single concept per test

copy iconCopydownload iconDownload
import assert from "assert";

describe("MomentJS", () => {
  it("handles date boundaries", () => {
    let date;

    date = new MomentJS("1/1/2015");
    date.addDays(30);
    assert.equal("1/31/2015", date);

    date = new MomentJS("2/1/2016");
    date.addDays(28);
    assert.equal("02/29/2016", date);

    date = new MomentJS("2/1/2015");
    date.addDays(28);
    assert.equal("03/01/2015", date);
  });
});

Use Promises, not callbacks

copy iconCopydownload iconDownload
import { get } from "request";
import { writeFile } from "fs";

get(
  "https://en.wikipedia.org/wiki/Robert_Cecil_Martin",
  (requestErr, response, body) => {
    if (requestErr) {
      console.error(requestErr);
    } else {
      writeFile("article.html", body, writeErr => {
        if (writeErr) {
          console.error(writeErr);
        } else {
          console.log("File written");
        }
      });
    }
  }
);

Async/Await are even cleaner than Promises

copy iconCopydownload iconDownload
import { get } from "request-promise";
import { writeFile } from "fs-extra";

get("https://en.wikipedia.org/wiki/Robert_Cecil_Martin")
  .then(body => {
    return writeFile("article.html", body);
  })
  .then(() => {
    console.log("File written");
  })
  .catch(err => {
    console.error(err);
  });

Don't ignore caught errors

copy iconCopydownload iconDownload
try {
  functionThatMightThrow();
} catch (error) {
  console.log(error);
}

Don't ignore rejected promises

copy iconCopydownload iconDownload
getdata()
  .then(data => {
    functionThatMightThrow(data);
  })
  .catch(error => {
    console.log(error);
  });

Use consistent capitalization

copy iconCopydownload iconDownload
const DAYS_IN_WEEK = 7;
const daysInMonth = 30;

const songs = ["Back In Black", "Stairway to Heaven", "Hey Jude"];
const Artists = ["ACDC", "Led Zeppelin", "The Beatles"];

function eraseDatabase() {}
function restore_database() {}

class animal {}
class Alpaca {}

Function callers and callees should be close

copy iconCopydownload iconDownload
class PerformanceReview {
  constructor(employee) {
    this.employee = employee;
  }

  lookupPeers() {
    return db.lookup(this.employee, "peers");
  }

  lookupManager() {
    return db.lookup(this.employee, "manager");
  }

  getPeerReviews() {
    const peers = this.lookupPeers();
    // ...
  }

  perfReview() {
    this.getPeerReviews();
    this.getManagerReview();
    this.getSelfReview();
  }

  getManagerReview() {
    const manager = this.lookupManager();
  }

  getSelfReview() {
    // ...
  }
}

const review = new PerformanceReview(employee);
review.perfReview();

Only comment things that have business logic complexity.

copy iconCopydownload iconDownload
function hashIt(data) {
  // The hash
  let hash = 0;

  // Length of string
  const length = data.length;

  // Loop through every character in data
  for (let i = 0; i < length; i++) {
    // Get character code.
    const char = data.charCodeAt(i);
    // Make the hash
    hash = (hash << 5) - hash + char;
    // Convert to 32-bit integer
    hash &= hash;
  }
}

Don't leave commented out code in your codebase

copy iconCopydownload iconDownload
doStuff();
// doOtherStuff();
// doSomeMoreStuff();
// doSoMuchStuff();

Don't have journal comments

copy iconCopydownload iconDownload
/**
 * 2016-12-20: Removed monads, didn't understand them (RM)
 * 2016-10-01: Improved using special monads (JP)
 * 2016-02-03: Removed type-checking (LI)
 * 2015-03-14: Added combine with type-checking (JR)
 */
function combine(a, b) {
  return a + b;
}

Avoid positional markers

copy iconCopydownload iconDownload
////////////////////////////////////////////////////////////////////////////////
// Scope Model Instantiation
////////////////////////////////////////////////////////////////////////////////
$scope.model = {
  menu: "foo",
  nav: "bar"
};

////////////////////////////////////////////////////////////////////////////////
// Action setup
////////////////////////////////////////////////////////////////////////////////
const actions = function() {
  // ...
};

Do you need background workers or multiple threads to fire multiple Async HttpWebRequests?

copy iconCopydownload iconDownload
var resultTask = Task.WhenAll(
    inputLines.Select(line => CallPsiForPrimaryStats(line)).ToArray());
-----------------------
static void Main(string[] args) { MainAsync(args).Wait(); }
static async Task MainAsync(string[] args)
{
  ...
  var tasks = inputLines.Select(line => CallPsiForPrimaryStats(line));
  var outputCache = await Task.WhenAll(tasks);
  await WriteCharacters(outputCache, outputFilePath);
  ...
private static async Task<string> CallPsiForPrimaryStats(string url)
{
  ...
  Task<WebResponse> task = Task.Factory.FromAsync(
      myReq.BeginGetResponse,
      myReq.EndGetResponse,
      (object)null);
  var result = await task;
  return ReadStreamFromResponse(result);
}
-----------------------
static void Main(string[] args) { MainAsync(args).Wait(); }
static async Task MainAsync(string[] args)
{
  ...
  var tasks = inputLines.Select(line => CallPsiForPrimaryStats(line));
  var outputCache = await Task.WhenAll(tasks);
  await WriteCharacters(outputCache, outputFilePath);
  ...
private static async Task<string> CallPsiForPrimaryStats(string url)
{
  ...
  Task<WebResponse> task = Task.Factory.FromAsync(
      myReq.BeginGetResponse,
      myReq.EndGetResponse,
      (object)null);
  var result = await task;
  return ReadStreamFromResponse(result);
}
-----------------------
static void Main(string[] args) { MainAsync(args).Wait(); }
static async Task MainAsync(string[] args)
{
  ...
  var tasks = inputLines.Select(line => CallPsiForPrimaryStats(line));
  var outputCache = await Task.WhenAll(tasks);
  await WriteCharacters(outputCache, outputFilePath);
  ...
private static async Task<string> CallPsiForPrimaryStats(string url)
{
  ...
  Task<WebResponse> task = Task.Factory.FromAsync(
      myReq.BeginGetResponse,
      myReq.EndGetResponse,
      (object)null);
  var result = await task;
  return ReadStreamFromResponse(result);
}

Community Discussions

Trending Discussions on clean-code-javascript
  • Do you need background workers or multiple threads to fire multiple Async HttpWebRequests?
Trending Discussions on clean-code-javascript

QUESTION

Do you need background workers or multiple threads to fire multiple Async HttpWebRequests?

Asked 2017-Jul-12 at 21:00

Overall goal

I'm trying to call to the Google PageSpeed Insights API with mutliple input urls read from a .txt file and to output the results to a .csv.

What I tried

I wrote a console app to try to fire these requests off, and then as they come back to add them to a list, and when they are all done, to write the list to the .csv file (async got a little nutty when trying to write the responses immediately to the .csv).

My code it below, and far from optimized. I come form a JavaScript background, where I usually don't use web workers or any other managed new threads, so I was trying to do the same in C#.

  1. Can I run do these multiple WebRequests and write them to a collection (or output file) without using multiple threads and have them all run in parallel, not having to wait for each request to come back before handling the next one?
  2. Is there a cleaner way to do this with callbacks?
  3. If threads or BackgroundWorkers are needed, what's a Clean Code way of doing this?

Initial Example Code

static void Main(string[] args)
{
    Console.WriteLine("Begin Google PageSpeed Insights!");

    appMode = ConfigurationManager.AppSettings["ApplicationMode"];
    var inputFilePath = READ_WRITE_PATH + ConfigurationManager.AppSettings["InputFile"];
    var outputFilePath = READ_WRITE_PATH + ConfigurationManager.AppSettings["OutputFile"];

    var inputLines = File.ReadAllLines(inputFilePath).ToList();

    if (File.Exists(outputFilePath))
    {
        File.Delete(outputFilePath);
    }

    List<string> outputCache = new List<string>();

    foreach (var line in inputLines)
    {
        var requestDataFromPsi = CallPsiForPrimaryStats(line);
        Console.WriteLine($"Got response of {requestDataFromPsi.Result}");

        outputCache.Add(requestDataFromPsi.Result);
    }

    var writeTask = WriteCharacters(outputCache, outputFilePath);

    writeTask.Wait();

    Console.WriteLine("End Google PageSpeed Insights");
}

private static async Task<string> CallPsiForPrimaryStats(string url)
{
    HttpWebRequest myReq = (HttpWebRequest)WebRequest.Create($"https://www.googleapis.com/pagespeedonline/v2/runPagespeed?url={url}&strategy=mobile&key={API_KEY}");
    myReq.Method = WebRequestMethods.Http.Get;
    myReq.Timeout = 60000;
    myReq.Proxy = null;
    myReq.ContentType = "application/json";

    Task<WebResponse> task = Task.Factory.FromAsync(
            myReq.BeginGetResponse,
            asyncResult => myReq.EndGetResponse(asyncResult),
            (object)null);

    return await task.ContinueWith(t => ReadStreamFromResponse(t.Result));
}

private static string ReadStreamFromResponse(WebResponse response)
{
   using (Stream responseStream = response.GetResponseStream())
   using (StreamReader sr = new StreamReader(responseStream))
   {
       string jsonResponse = sr.ReadToEnd();
       dynamic jsonObj = Newtonsoft.Json.JsonConvert.DeserializeObject(jsonResponse);

       var psiResp = new PsiResponse()
       {
           Url = jsonObj.id,
           SpeedScore = jsonObj.ruleGroups.SPEED.score,
           UsabilityScore = jsonObj.ruleGroups.USABILITY.score,
           NumberResources = jsonObj.pageStats.numberResources,
           NumberHosts = jsonObj.pageStats.numberHosts,
           TotalRequestBytes = jsonObj.pageStats.totalRequestBytes,
           NumberStaticResources = jsonObj.pageStats.numberStaticResources,
           HtmlResponseBytes = jsonObj.pageStats.htmlResponseBytes,
           CssResponseBytes = jsonObj.pageStats.cssResponseBytes,
           ImageResponseBytes = jsonObj.pageStats.imageResponseBytes,
           JavascriptResponseBytes = jsonObj.pageStats.javascriptResponseBytes,
            OtherResponseBytes = jsonObj.pageStats.otherResponseBytes,
            NumberJsResources = jsonObj.pageStats.numberJsResources,
            NumberCssResources = jsonObj.pageStats.numberCssResources,

        };
        return CreateOutputString(psiResp);
    }
}

static async Task WriteCharacters(List<string> inputs, string outputFilePath)
{
    using (StreamWriter fileWriter = new StreamWriter(outputFilePath))
    {
        await fileWriter.WriteLineAsync(TABLE_HEADER);

        foreach (var input in inputs)
        {
            await fileWriter.WriteLineAsync(input);
        }
    }
}

private static string CreateOutputString(PsiResponse psiResponse)
{
    var stringToWrite = "";

    foreach (var prop in psiResponse.GetType().GetProperties())
    {
        stringToWrite += $"{prop.GetValue(psiResponse, null)},";
    }
    Console.WriteLine(stringToWrite);
    return stringToWrite;
}

Update: After Refactor from Stephen Cleary Tips

Problem is this still runs slow. The original took 20 minutes, and after refactor it still took 20 minutes. It seems to be throttled somewhere, maybe by Google on the PageSpeed API. I tested it, calling calling https://www.google.com/, https://www.yahoo.com/, https://www.bing.com/ and 18 others and it runs slowly as well, having a bottleneck of only processing about 5 requests at a time. I tried refactoring to run 5 test URLs and then write to file and repeat but it only marginally sped up the process.

static void Main(string[] args) { MainAsync(args).Wait(); }
static async Task MainAsync(string[] args)
{
    Console.WriteLine("Begin Google PageSpeed Insights!");

    appMode = ConfigurationManager.AppSettings["ApplicationMode"];
    var inputFilePath = READ_WRITE_PATH + ConfigurationManager.AppSettings["InputFile"];
    var outputFilePath = READ_WRITE_PATH + ConfigurationManager.AppSettings["OutputFile"];

    var inputLines = File.ReadAllLines(inputFilePath).ToList();

    if (File.Exists(outputFilePath))
    {
        File.Delete(outputFilePath);
    }

    var tasks = inputLines.Select(line => CallPsiForPrimaryStats(line));
    var outputCache = await Task.WhenAll(tasks);

    await WriteCharacters(outputCache, outputFilePath);

    Console.WriteLine("End Google PageSpeed Insights");
}

private static async Task<string> CallPsiForPrimaryStats(string url)
{
    HttpWebRequest myReq = (HttpWebRequest)WebRequest.Create($"https://www.googleapis.com/pagespeedonline/v2/runPagespeed?url={url}&strategy=mobile&key={API_KEY}");
    myReq.Method = WebRequestMethods.Http.Get;
    myReq.Timeout = 60000;
    myReq.Proxy = null;
    myReq.ContentType = "application/json";
    Console.WriteLine($"Start call: {url}");

    // Try using `HttpClient()` later
    //var myReq2 = new HttpClient();
    //await myReq2.GetAsync($"https://www.googleapis.com/pagespeedonline/v2/runPagespeed?url={url}&strategy=mobile&key={API_KEY}");

    Task<WebResponse> task = Task.Factory.FromAsync(
        myReq.BeginGetResponse,
        myReq.EndGetResponse,
        (object)null);
    var result = await task;
    return ReadStreamFromResponse(result);
}

private static string ReadStreamFromResponse(WebResponse response)
{
    using (Stream responseStream = response.GetResponseStream())
    using (StreamReader sr = new StreamReader(responseStream))
    {
        string jsonResponse = sr.ReadToEnd();
        dynamic jsonObj = Newtonsoft.Json.JsonConvert.DeserializeObject(jsonResponse);

        var psiResp = new PsiResponse()
        {
            Url = jsonObj.id,
            SpeedScore = jsonObj.ruleGroups.SPEED.score,
            UsabilityScore = jsonObj.ruleGroups.USABILITY.score,
            NumberResources = jsonObj.pageStats.numberResources,
            NumberHosts = jsonObj.pageStats.numberHosts,
            TotalRequestBytes = jsonObj.pageStats.totalRequestBytes,
            NumberStaticResources = jsonObj.pageStats.numberStaticResources,
            HtmlResponseBytes = jsonObj.pageStats.htmlResponseBytes,
            CssResponseBytes = jsonObj.pageStats.cssResponseBytes,
            ImageResponseBytes = jsonObj.pageStats.imageResponseBytes,
            JavascriptResponseBytes = jsonObj.pageStats.javascriptResponseBytes,
            OtherResponseBytes = jsonObj.pageStats.otherResponseBytes,
            NumberJsResources = jsonObj.pageStats.numberJsResources,
            NumberCssResources = jsonObj.pageStats.numberCssResources,

        };
        return CreateOutputString(psiResp);
    }
}

static async Task WriteCharacters(IEnumerable<string> inputs, string outputFilePath)
{
    using (StreamWriter fileWriter = new StreamWriter(outputFilePath))
    {
        await fileWriter.WriteLineAsync(TABLE_HEADER);

        foreach (var input in inputs)
        {
            await fileWriter.WriteLineAsync(input);
        }
    }
}

private static string CreateOutputString(PsiResponse psiResponse)
{
    var stringToWrite = "";
    foreach (var prop in psiResponse.GetType().GetProperties())
    {
        stringToWrite += $"{prop.GetValue(psiResponse, null)},";
    }
    Console.WriteLine(stringToWrite);
    return stringToWrite;
}

ANSWER

Answered 2017-Jul-12 at 14:16

Can I run do these multiple WebRequests and write them to a collection (or output file) without using multiple threads and have them all run in parallel, not having to wait for each request to come back before handling the next one?

Sure thing.

Is there a cleaner way to do this with callbacks?

You can always iterate over the input lines and grab a collection of the tasks that are all running.

var resultTask = Task.WhenAll(
    inputLines.Select(line => CallPsiForPrimaryStats(line)).ToArray());

This is analogous to using the Q library for promises in Javascript. With .Net tasks the host machine will spin up as many of the processes as it can in parallel.

The resultTask will be a collection of the results you can work with, much like your outputCache.

In the code you added above, the call to .Result in the loop will be synchronous. Nothing is happening in parallel. Be careful when waiting for all of this, you might run out of memory before it's all passed back! Might be worth streaming this to the file as they return, and having a semaphore or lock prevent them all writing to the stream at once.

Also I think the WebClient class is more idiomatic these days than hand rolling HttpWebRequest.

If threads or BackgroundWorkers are needed, what's a Clean Code way of doing this?

This is the beauty of the Task library and .Net's async stack. You shouldn't need to do anything with threads.

It's important to know the difference between an async/await type call and a synchronous call. Anywhere you see async in the method declaration and await in the body means that the code is freeing up the current synchronisation thread to do other work, like fire off more tasks. When you see .Result or .Wait() these are synchronous and are therefore blocking the main synchronisation thread. Which means no ability for easy parallelism.

Source https://stackoverflow.com/questions/45059008

Community Discussions, Code Snippets contain sources that include Stack Exchange Network

Vulnerabilities

No vulnerabilities reported

Install clean-code-javascript

You can download it from GitHub.

Support

For any new features, suggestions and bugs create an issue on GitHub. If you have any questions check and ask questions on community page Stack Overflow .

DOWNLOAD this Library from

Find, review, and download reusable Libraries, Code Snippets, Cloud APIs from
over 430 million Knowledge Items
Find more libraries
Reuse Solution Kits and Libraries Curated by Popular Use Cases

Save this library and start creating your kit

Explore Related Topics

Share this Page

share link
Reuse Pre-built Kits with clean-code-javascript
Compare Architecture Libraries with Permissive License
Find, review, and download reusable Libraries, Code Snippets, Cloud APIs from
over 430 million Knowledge Items
Find more libraries
Reuse Solution Kits and Libraries Curated by Popular Use Cases

Save this library and start creating your kit

  • © 2022 Open Weaver Inc.