system-design-primer | design large-scale systems | Architecture library
kandi X-RAY | system-design-primer Summary
kandi X-RAY | system-design-primer Summary
Learn how to design large-scale systems. Prep for the system design interview. Includes Anki flashcards.
Support
Quality
Security
License
Reuse
Top functions reviewed by kandi - BETA
- Dispatch a call
- Dispatches the given call
- Set a call to the employee
- Adds the results to the cache
- Get a node matching the query
- Process a query
- Parse a query
- Crawl all pages
- Crawl a page
- Disks a vehicle
- Return the value associated with the given key
- Removes a key from the table
- Return the card value
- Calculate the score of this node
- Add a key to the table
- Reduce budget notifications
- Generator for mapper
- Mapper a line of time series
- Given a line return the product product category
- Remove the vehicle from the spot
system-design-primer Key Features
system-design-primer Examples and Code Snippets
Latency Comparison Numbers
--------------------------
L1 cache reference 0.5 ns
Branch mispredict 5 ns
L2 cache reference 7 ns 14x L1 cache
Mutex
Latency Comparison Numbers
--------------------------
L1 cache reference 0.5 ns
Branch mispredict 5 ns
L2 cache reference 7 ns 14x L1 cache
Mutex
Power Exact Value Approx Value Bytes
---------------------------------------------------------------
7 128
8 256
10 1024 1 thousand
Community Discussions
Trending Discussions on system-design-primer
QUESTION
I'm looking at donne martin's design for a web crawler. the crawler service processes a newly crawled url, and then:
- Adds a job to the Reverse Index Service queue to generate a reverse index
- Adds a job to the Document Service queue to generate a static title and snippet
what would happen if instead the crawler service would synchronously call these 2 services? I would still be able to horizontally scale all 3 services according to the load on each, right? what came to me as a possible reason is just more complex flow control if one of them fails. are there other more compelling reasons for these async jobs?
...ANSWER
Answered 2020-Apr-10 at 03:01There are likely more reasons behind this design choice, but one is almost certainly use of Microservices. It is a popular technique, so demonstrating command of it is a good idea for answering design questions and benefits of it are well described on Wikipedia:
- Modularity: This makes the application easier to understand, develop, test, and become more resilient to architecture erosion.[6] This benefit is often argued in comparison to the complexity of monolithic architectures.[33]
- Scalability: Since microservices are implemented and deployed independently of each other, i.e. they run within independent processes, they can be monitored and scaled independently.[34]
- Integration of heterogeneous and legacy systems: microservices is considered as a viable mean for modernizing existing monolithic software application.[35][36] There are experience reports of several companies who have successfully replaced (parts of) their existing software by microservices, or are in the process of doing so.[37] The process for Software modernization of legacy applications is done using an incremental approach.[38]
- Distributed development: it parallelizes development by enabling small autonomous teams to develop, deploy and scale their respective services independently.[39] It also allows the architecture of an individual service to emerge through continuous refactoring.[40] Microservice-based architectures facilitate continuous integration, continuous delivery and deployment.[41] [42]
All of those apply in this case. Indeed, well-defined API makes the modules separate, reusable, easy to understand. Most likely each of the 3 modules will have very different execution time and CPU/memory requirements, so scaling them separately makes a lot of sense. Some companies like Amazon mentioned on the page might go much further splitting those modules into microservices based on the team number, so this split into 3 services can very well be chosen based on the assumption of having 3 teams, rather than technical constraints.
The page also describes criticism of the technique.
Community Discussions, Code Snippets contain sources that include Stack Exchange Network
Vulnerabilities
No vulnerabilities reported
Install system-design-primer
You can use system-design-primer like any standard Python library. You will need to make sure that you have a development environment consisting of a Python distribution including header files, a compiler, pip, and git installed. Make sure that your pip, setuptools, and wheel are up to date. When using pip it is generally recommended to install packages in a virtual environment to avoid changes to the system.
Support
Reuse Trending Solutions
Find, review, and download reusable Libraries, Code Snippets, Cloud APIs from over 650 million Knowledge Items
Find more librariesStay Updated
Subscribe to our newsletter for trending solutions and developer bootcamps
Share this Page