Cloud computing is the on-demand availability of computer system resources, especially data storage (cloud storage) and computing power, without direct active management by the user. The term is generally used to describe data centers available to many users over the Internet. Large clouds, predominant today, often have functions distributed over multiple locations from central servers. If the connection to the user is relatively close, it may be designated an edge server.

AWS, Azure, and Google Cloud are the leading public vendors in cloud computing, offering a myriad of services and functionalities. The open-source community is heavily involved in this cloud trend. Open-source technologies provide the foundation for many public cloud services, and many enterprises are using these same technologies to build private clouds and hybrid clouds. In addition, many open-source projects offer cloud-based software-as-a-service (SaaS) versions of their applications.

Popular New Releases in Cloud

localstack

LocalStack release 0.14.2

microservices-demo

v0.3.6

sops

v3.7.2

go-cloud

v0.24.0

cortex

v0.42.0

Popular Libraries in Cloud

localstack

by localstack doticonpythondoticon

star image 40029 doticonNOASSERTION

πŸ’» A fully functional local AWS cloud stack. Develop and test your cloud & Serverless apps offline!

og-aws

by open-guides doticonshelldoticon

star image 29988 doticonCC-BY-4.0

πŸ“™ Amazon Web Services β€” a practical guide

aws-cli

by aws doticonpythondoticon

star image 12274 doticonNOASSERTION

Universal Command Line Interface for Amazon Web Services

microservices-demo

by GoogleCloudPlatform doticonpythondoticon

star image 12028 doticonApache-2.0

Sample cloud-native application with 10 microservices showcasing Kubernetes, Istio, gRPC and OpenCensus.

awesome-kubernetes

by ramitsurana doticonshelldoticon

star image 11943 doticonNOASSERTION

A curated list for awesome kubernetes sources :ship::tada:

azure-quickstart-templates

by Azure doticonshelldoticon

star image 11650 doticonMIT

Azure Quickstart Templates

awesome-aws

by donnemartin doticonpythondoticon

star image 9608 doticonNOASSERTION

A curated list of awesome Amazon Web Services (AWS) libraries, open source repos, guides, blogs, and other resources. Featuring the Fiery Meter of AWSome.

sops

by mozilla doticongodoticon

star image 9259 doticonMPL-2.0

Simple and flexible tool for managing secrets

amplify-js

by aws-amplify doticontypescriptdoticon

star image 8675 doticonApache-2.0

A declarative JavaScript library for application development using cloud services.

Trending New libraries in Cloud

infracost

by infracost doticongodoticon

star image 6374 doticonApache-2.0

Cloud cost estimates for Terraform in pull requestsπŸ’°πŸ“‰ Love your cloud bill!

CBL-Mariner

by microsoft doticongodoticon

star image 3213 doticonMIT

Linux OS for Azure 1P services and edge appliances

AZ-204-DevelopingSolutionsforMicrosoftAzure

by MicrosoftLearning doticoncsharpdoticon

star image 1391 doticonMIT

AZ-204: Developing solutions for Microsoft Azure

threadx

by azure-rtos doticoncdoticon

star image 1280 doticonNOASSERTION

Azure RTOS ThreadX is an advanced real-time operating system (RTOS) designed specifically for deeply embedded applications.

AZ-104-MicrosoftAzureAdministrator

by MicrosoftLearning doticonpowershelldoticon

star image 1258 doticonMIT

AZ-104 Microsoft Azure Administrator

eks-distro

by aws doticonshelldoticon

star image 1100 doticonApache-2.0

Amazon EKS Distro (EKS-D) is a Kubernetes distribution based on and used by Amazon Elastic Kubernetes Service (EKS) to create reliable and secure Kubernetes clusters.

Stormspotter

by Azure doticonpythondoticon

star image 1024 doticonNOASSERTION

Azure Red Team tool for graphing Azure and Azure Active Directory objects

leapp

by Noovolari doticontypescriptdoticon

star image 941 doticonMPL-2.0

Leapp is the DevTool to access your cloud

external-secrets

by external-secrets doticongodoticon

star image 914 doticonApache-2.0

External Secrets Operator reads information from a third-party service like AWS Secrets Manager and automatically injects the values as Kubernetes Secrets.

Top Authors in Cloud

1

Azure-Samples

418 Libraries

star icon13271

2

aws-samples

413 Libraries

star icon20599

3

Azure

404 Libraries

star icon68389

4

microsoft

252 Libraries

star icon23913

5

awslabs

154 Libraries

star icon29934

6

heroku

130 Libraries

star icon10583

7

GoogleCloudPlatform

115 Libraries

star icon38623

8

aws

86 Libraries

star icon58337

9

MicrosoftDocs

84 Libraries

star icon11461

10

amazon-archives

64 Libraries

star icon9170

1

418 Libraries

star icon13271

2

413 Libraries

star icon20599

3

404 Libraries

star icon68389

4

252 Libraries

star icon23913

5

154 Libraries

star icon29934

6

130 Libraries

star icon10583

7

115 Libraries

star icon38623

8

86 Libraries

star icon58337

9

84 Libraries

star icon11461

10

64 Libraries

star icon9170

Trending Kits in Cloud

Developing and deploying apps, managing infrastructure, monitoring, logging activity, and other development and operations procedures can all be automated using C-Sharp DevOps libraries. The programming language C# is frequently used to create various applications, including backend services, desktop, mobile, and web based. On the .NET platform, which offers a wealth of libraries and tools for creating, deploying, and maintaining applications, C# serves as the principal language for developing applications.  


Building and deploying apps, managing infrastructure, and monitoring and logging are just a few development and operations processes that can be automated in DevOps using C# libraries. By automating and streamlining their operations, these libraries can aid developers, and DevOps engineers in producing higher-quality software more rapidly and effectively. DevOps aims to make it possible for businesses to produce higher-quality, more reliable, and quicker software. A software development process called DevOps strongly emphasizes teamwork, communication, and integration between IT operations teams and software developers. Azure Microsoft offers a suite of DevOps tools and services for creating, delivering, and managing applications. It is a cloud-based platform that interfaces with various tools and services, such as GitHub, Azure, and Visual Studio, to offer a full range of DevOps capabilities. 


Check out the below list to find the best C# DevOps libraries for your application: 

Cloud computing Module 1:


The platforms that provides on-demand computing resources from application to data server to clients over the internet for a fee per use basis.

It's composed of:

5 essential characteristics are:

  1. On-demand self-service: Clients are provided multiple services such as application, servers, storages etc.
  2. Broad network access: Client is given access to all these resources using network.
  3. Resource pooling: Gives providers economies of scale, which they pass on to their customers making cloud cost effective. Resources are dynamically assigned and reassigned.
  4. Rapid Elasticity: You can use more resources when you need them, and scale back when you don't.
  5. Measured services: You only pay for what you use or reserve.

3 Deployment models:

  1. Public: Is when you use cloud services on open internet on cloud provider's hardware, but its usage is shared by other companies.
  2. Private: Is when cloud services are used by a single organization. It could run on premises or owned, managed by service providers.
  3. Hybrid: Is when you use the mixture of public and private cloud.

3 service models:

  1. Infrastructure (IaaS) Infrastructure as a Services: You are provided with a complete infrastructure and physical resources such as networking, storages or servers without the worry to manage them.
  2. Platform (PaaS): clients are provided with hardware and software tools required for developing and deploying application to users.
  3. Application (SaaS): also known as on-demand software, where a subscription based application are hosted and provided to those who wish to pay for them or sometimes for free.

Pay-as-you-go: pay per use.

Key drivers for organization to move into cloud computing are:

agility, flexibility and competitiveness.

Key consideration for organizations:

Infrastructure and workload: The cost of building data center and purchasing servers can be reduced using cloud computing as you only need to pay-as-you-go.

SaaS and Development Platforms: Paying for application access costs less than buying COTS applications and later paying for upgrades etc.

Speed and productivity: getting the server to run in just few hours VS waiting for week for traditional server/platforms.

Benefits of Cloud adoption:

Flexibility:

Scale services up and down as per need, get customized applications, and access to resources from anywhere. It also provide security using private clouds, encryptions and API keys.

Efficiency:

You can get application quickly to market and not worry about its infrastructure or maintenance, also you can access it from any internet-connected devices and no data loss due to hardware failure.

strategic value:

Provides competitive advantage by providing most innovative technology to clients.

Challenges of cloud adoption:

Data security: loss or unavailability of data can cause business disruption.

governance, legal and regulatory issues

problem in choosing right deployment and service model for specific needs

partnering with right cloud providers.

Some cloud service providers:

Alibaba cloud (aliyun)

AWS amazon web services

Google cloud platform (GCP)

IBM cloud

Microsoft azure

Oracle cloud etc.


Business Case for Cloud:

The adoption of cloud technologies is enabling enterprises, big and small, to be agile, innovative, and competitive, and to create differentiated customer experiences. The question organizations are asking is not whether they should move to the cloud, rather what strategy they should adopt to move to the cloud.

Some case studies that demonstrate the impact businesses have created by adopting cloud:

  • American Airlines adopting cloud technologies to deliver customer value rapidly across its enterprise
  • UBank leveraging cloud platform services to give more control to their developers thereby removing barriers to innovation
  • Bitly leveraging the scalability offered by cloud infrastructure for low-latency delivery to its geographically disbursed enterprise customers
  • ActivTrades leveraging the infrastructure, storage, network, and security offerings on the cloud to accelerate execution and delivery of new functions in their online trading systems to their customers


Emerging Technologies accelerated by cloud:

Emerging technologies, powered by the cloud, are disrupting existing business models and creating unprecedented opportunities for businesses to grow, innovate, and create value for their customers.

Some case studies that demonstrate how the use of emerging technologies on the cloud is creating value for millions around the world:

  • The use of the Internet of Things on the cloud to combat poaching of endangered rhinos in South Africa
  • Artificial Intelligence on the cloud being leveraged to deliver unique digital experiences to millions of fans around the world by the United States Tennis Association
  • Blockchain on the cloud helping farmers reduce waste by building traceability and transparency in the food supply chain
  • The use of data analytics for driving predictive maintenance solutions for a city’s infrastructure by KONE

Cloud Computing Module 2:

Discusses service models:

IaaS: Provides networking, storage and computing resources to customer on pay-per-use basis. It also provides virtualization and hypervisor layer.

  1. Here customer can use VMs that provides hypervisor, OSs and you can also install middleware, applications of your choice.
  2. You can create a storage of you choice for backups and workloads.
  3. It also provides tracking and monitoring the performance and usage of cloud resources and disaster recovery.
  4. It helps organization in setting up test and development environment faster, and focus on productivity than on managing infrastructure.
  5. It also provides business continuity and disaster recovery.
  6. Also helps in fast deployment and scaling.
  7. Also helps in data analysis.
  8. Fastest growing cloud model today.

Concerns: Lack of transparency and dependency on third-party.

PaaS: Provides customers with a platform to manage, develop and deploy software without having to worry about the inner infrastructure.

Advantages:

  • Scalability
  • Faster time to market
  • Agility

Risks:

  • Cybersecurity threats
  • Dependency on platform providers
  • Lack of control over changing services etc.

Use cases:

  • IoT
  • API development
  • Business analytics

Essential characteristics:

  • Run-time environment
  • Support services and APIs to help in development
  • Rapid deployment
  • High level of abstraction.

SaaS: provides customers to access cloud-based software against a subscription fee.

everything is managed by provides such as security, servers, databases etc.

  • Benefits:
  • reduces time from decision to value
  • increase efficiency and productivity

Use Case:

  • Minimal input
  • Avoid upgrades, maintenance
  • Reduce on-premises IT infrastructure

Concerns:

  • Data safety and ownership
  • third-party maintaining critical data
  • require internet connection all time.

Deployment models:

Indicate where the infrastructure resides, who owns and manages it, and how cloud resources and services are made available to users. There are three main deployment models available on the cloudβ€”Public, Private, and Hybrid.

Public cloud model, the service provider owns, manages, provisions, and maintains the physical infrastructure such as data centers, servers, networking equipment, and storage, with users accessing virtualized compute, networking and storage resources as services.

Advantages:

  • On-demand resources
  • Economies of scale
  • Highly reliable

Concerns:

  • Security
  • Data sovereignty compliance

Use cases:

  • Reduce time-to-market
  • Fluctuating resource needs
  • Backup for disaster recovery etc.
  • greater accessibility


Private cloud model, the provider provisions the cloud infrastructure for exclusive use by a single organization. The private cloud infrastructure can be internal to the organization and run or on-premises. Or, it can be on a public cloud, as in case of Virtual Private Clouds (VPC){a private computing environment in a shared public cloud} and be owned, managed, and operated by the cloud provider.

Benefits:

  • Scalability
  • Internal control
  • Reduced cost
  • Controlled access and security


Use Case:

  • Data and service import
  • portability in software devolopment
  • full control over critical security


Hybrid cloud model, an organization’s on-premise private cloud and third-party, public cloud is connected as a single, flexible infrastructure leveraging the features and benefits of both Public and Private clouds.

Types:

Mono: single provider

Multi: can be deployed on any public cloud infrastructure

composite: can move single application over multiple providers, thus greater flexibility.

  • Benefits:
  • Security and compliance
  • Scalability and resilience
  • Resource optimization
  • Cost Saving


Module 3 Cloud infrastructure:

Overview:

  • Customers need to plan the infrastructure architecture after choosing the cloud service model and type offered by vendors.
  • The infrastructure layer consists of physical resources housed in Regions, Zones, and Data Centers.
  • Cloud providers’ IT environment is distributed across many regions worldwide, and each region can have multiple Zones.
  • Availability Zones (AZs) are distinct Data Centers with their own resources, which improves fault tolerance and decreases latency.
  • Cloud providers offer several compute options: Virtual Servers, Bare Metal Servers, and Server-less computing resources.
  • Information and data can be stored in many different types of storage options on the Cloud, including Block, File, and Object storage.
  • Networking infrastructure in a cloud data center includes traditional networking hardware, router and switches, but also Software Defined Networking (SDN) options for easier network provisioning and management.
  • Public and private network interfaces need to be set up when provisioning servers in the cloud.
  • Security measures like Security Groups and Access Control Lists (ACLs) can be set up to control access to resources in the cloud.


Virtualization and Hypervisors:

Virtualization and its importance in cloud computing. It defines virtualization as the process of creating a virtual version of something using a hypervisor. There are two main types of hypervisors, Type 1 and Type 2, with Type 1 being the most secure and frequently used. Virtual machines (VMs) are software-based computers that are independent of one another and can run on a hypervisor. VMs are portable and can be moved from one hypervisor to another almost instantly. The key benefits of virtualization include cost savings, agility and speed, and lower downtime. Virtualization and VMs are essential components of cloud computing.

Virtual Machine and its Types:

  • Virtual Machines (VMs) are known by various names depending on the cloud provider.
  • Cloud providers offer VMs in various configurations and deployment options to cater to different use cases.
  • When creating a virtual server, you specify the region, zone or data center, operating system, shared or dedicated VMs, and storage and networking options.
  • Cloud providers offer predefined sizes and configurations for VMs ranging from single virtual core to multiple virtual cores with larger amounts of RAM to cater to different workloads.
  • Public or shared cloud VMs are provider-managed, multi-tenant deployments that can be provisioned on-demand with predefined sizes.
  • Transient or spot VMs take advantage of unused capacity in a cloud data center, are available at a discount, and are suitable for non-production workloads.
  • Reserved virtual server instances allow you to reserve capacity and guarantee resources for future deployments and can lower costs compared to hourly or monthly instances.
  • Dedicated hosts offer single-tenant isolation, maximum control over workload placement, and are used to meet compliance, regulatory requirements, or specific licensing terms.

Bare metal server:

  • Bare metal server is a single-tenant, dedicated physical server that is managed by the cloud provider.
  • Customer is responsible for administering and managing everything except OS on the server.
  • Bare metal servers can be preconfigured or custom-configured as per customer specifications, including the processors, RAM, hard drives, specialized components, and the OS.
  • Bare metal servers are fully customizable, expensive and intended for long-term, high-performance use in highly secure and isolated environments.
  • Customers have full access and control of bare metal servers, which can be used for high-performance computing or data intense applications that require minimal latency-related delays.
  • Bare metal servers work best for CPU and I/O intensive workloads, excel with highest performance and security, satisfy strict compliance requirements, and offer complete flexibility, control and transparency but come with added management and operational overhead.
  • Virtual servers are rapidly provisioned, provide an elastic and scalable environment, and are low cost to use, but can be limited in throughput and performance due to sharing underlying hardware with other virtual servers.

Secure Networking in Cloud:

  • Building a secure cloud networking presence is crucial as cloud environments gain greater adoption and digital data invites rapidly increasing cybersecurity threats.
  • Building a cloud network is not much different from deploying a network in an on-premises data center, except that logical instances of networking elements are used in the cloud.
  • Networking functions are delivered as a service in the cloud, rather than in the form of rack-mounted devices.
  • Cloud networks are deployed in networking spaces that are logically separated segments of the networks using options such as Virtual private Cloud (VPC) that in turn can be divided into smaller segments called subnets.
  • Cloud resources, such as VMs or Virtual Server Instances (VSIs), storage, network connectivity, and load balancers are deployed into subnets.
  • Subnets are the main area where security is implemented in the cloud, with every subnet protected by Access Control Lists (ACLS) that serve as a subnet-level firewall.
  • Security Groups are used to provide security at the instance level, such as VSIs.
  • Public gateways are added to the network to enable user access to the application in the internet tier, while VPNs are used to securely connect on-premises resources to the cloud.
  • Load Balancers are used to ensure availability of bandwidth for different applications.
  • Dedicated high-speed connections between clouds and on-premises resources are a more secure and efficient way than public connectivity solutions.
  • Cloud network building entails creating a set of logical constructs that deliver networking functionality similar to data center networks.


Containers:

  • Containers are an executable unit of software that package application code, libraries, and dependencies in a common way to run anywhere, including desktop, traditional IT, and the cloud.
  • Containers are small, fast, and portable, and do not require a guest OS in every instance, unlike virtual machines.
  • Container technology has been around since 2008 when the Linux kernel introduced C groups and control groups.
  • Container technology includes Docker, Cloud Foundry, Rocket, and other container runtimes.
  • Containers solve compatibility issues that arise when applications developed on one machine are pushed into production on another.
  • The containerization process involves creating a manifest that describes the container, creating an image, and creating the container itself.
  • The container runs on a host operating system with a runtime engine instead of a hypervisor.
  • Containers are much more lightweight than virtual machines, making it easier to deploy multiple containers.

Cloud Storage:

Overview to cloud storage:

  • Cloud storage refers to saving data and files in the cloud.
  • Cloud providers manage and maintain the storage infrastructure to ensure data accessibility.
  • Cloud storage services allow for easy scalability and pay-as-you-go pricing.
  • There are four main types of cloud storage: Direct Attached, File Storage, Block Storage, and Object Storage.
  • Direct Attached storage is typically only used to store an operating system due to its limitations.
  • File Storage is presented to compute nodes over an ethernet network and can be mounted or used on multiple servers.
  • Block Storage is presented using high-speed fiber connections and is suitable for use with databases and applications where disk speed is important.
  • IOPS refers to the speed of the storage and how quickly data can be read from or written to it.
  • Persistence is a term used when provisioning File or Block storage and relates to what happens to the storage once the compute node it is attached to is terminated.
  • Snapshots are a way to backup data in the cloud and are fast to create, don't require downtime, and record only changes to the data.
  • Object Storage is accessed via an API and is the cheapest and slowest in terms of read and write speeds but is infinite in size to the end user, making it a great repository for unstructured data types.

File storage:

  1. File Storage is a type of storage that needs to be attached to a compute node before it can be accessed and used.
  2. File Storage is less expensive, more resilient to failure, and requires lesser disk management and maintenance than direct attached storage.
  3. File Storage can provision larger amounts of storage and present it as a disk to a server.
  4. File Storage is mounted from remote storage appliances, making it more secure and offering services such as encryption in transit and at rest.
  5. File Storage is mounted to compute nodes via an ethernet network, which can sometimes affect its speed or bandwidth.
  6. File Storage can typically be mounted onto more than one compute node at a time, making it an ideal solution for common storage requirements.
  7. IOPS capacity is an important consideration when provisioning File Storage, as it determines the speed at which disks can write and read data and can affect the performance of applications.
  8. File Storage is suitable for workloads where consistent high network speeds are not a requirement, such as departmental file shares or landing zones for incoming files.

Block storage:

  • Block storage breaks files into chunks or blocks of data and stores each block separately under a unique address.
  • Block storage must be attached to a compute node before it can be utilized for workloads.
  • Block storage can be mounted from remote storage appliances, making it resilient to failure and more secure due to encryption services available on these appliances.
  • Block storage is mounted as a volume to compute nodes using a dedicated fiber optic network, which is more expensive than the ethernet network used for file storage.
  • Block storage is suitable for workloads that need low-latency storage, such as databases and mail servers, but not for workloads requiring disk sharing between compute nodes.
  • IOPS capacity must be considered when provisioning block storage.
  • Both block and file storage are taken from appliances maintained by the service provider and often include data encryption at rest and in transit.
  • File storage is attached to compute nodes using an ethernet network and can be mounted onto multiple nodes at once.
  • File storage is a good choice where file shares are required and cost is a factor, or where workloads do not require lightning fast connectivity to storage.
  • Block storage is a good choice for applications that need consistent fast access to disk.
  • The next video will cover Object Storage.

Object storage:

  • Object Storage does not require connection to a compute node; data is managed through an API
  • Object Storage is less expensive than other cloud storage options
  • Object Storage is effectively infinite, as you only pay for the storage you use
  • Object Storage is great for storing large amounts of unstructured data
  • Buckets are used to store objects in a structurally flat way
  • Objects can have metadata added to them for locating and accessing the data
  • Service providers ensure sufficient storage capacity, resilience, and high availability
  • Object Storage is suitable for static data such as text files, audio and video files, IOT data, virtual machine images, backup files, and data archives
  • Object Storage is not suitable for running operating systems or applications such as databases
  • Multiple buckets can be used, but buckets cannot be nested within each other
  • Different types of buckets may be offered with different charges based on resilience, availability, and frequency of access.

CDN content delivery network:

  • A Content Delivery Network (CDN) is a distributed server network that delivers temporarily stored, or cached, copies of website content to users based on their geographic location.
  • CDN stores content in distributed locations, reducing the distance between website visitors and the website server.
  • The main benefit of a CDN is that it makes the website faster by reducing the amount of time it takes to deliver the content.
  • Users accessing a website from different parts of the world experience different round-trip times, which affects website speed.
  • CDN places content delivery network endpoints in as many locations around the world as possible to reduce the distance between the user and the content.
  • CDN retrieves content from the server and distributes it to users in different locations, reducing the amount of time it takes to retrieve the content.
  • The reduction in distance between the user and content not only benefits the end-user but also indirectly reduces the load on the server and increases uptime and security.

Module 4:

Hybrid and multi cloud

  • Hybrid Cloud connects an organization's private cloud and third-party public cloud into a single infrastructure for running applications
  • Multi-cloud is a cloud adoption strategy that embraces a mix of cloud models from different service providers
  • Hybrid Multi-cloud implies leveraging the best of cloud models and services across different service providers for seamless application and workload management
  • Use cases for hybrid or multi-cloud include cloud scaling, building a composite cloud, modernizing reservation systems in the airline industry, and using data and AI to improve airline maintenance and reduce delays
  • Cloud scaling allows businesses to scale up in response to user load and deprovision resources when no longer needed
  • Building a composite cloud involves composing an application across multiple cloud environments to scale up portions in response to specific events while keeping other portions scaled individually
  • Modernizing reservation systems involves creating new user experiences using mobile applications and cloud-based recommendation features
  • Using data and AI in the airline industry allows businesses to take advantage of historical data to improve maintenance and reduce delays

Microservices:

  • Microservices architecture is an approach in which a single application is composed of many loosely coupled and independently deployable, smaller components or services.
  • These services have their own stack running on their own containers and communicate with one another over a combination of APIs, event streaming, and message brokers.
  • Microservices enable developers to work more efficiently by developing and updating application components independently and using different stacks and runtime environments for different components.
  • Microservices also allow for independent scaling of components, reducing waste and cost associated with having to scale entire applications.
  • Cloud development platforms provide developers with an ecosystem of code that can be easily and securely integrated into applications.
  • Microservices break down large applications into their core functions and are developed independently of one another, yet work together on the cloud development platform to create a functioning application.
  • Microservices approach allows developers to quickly innovate applications in parallel and lets users focus on the things that really interest them.
  • Microservices help businesses keep up and grow with their customers.
  • The use case presented in the passage is about an online streaming media service called Dream Game that uses microservices to develop a better user experience for viewers.

Serverless:

  • Serverless computing offloads responsibility for infrastructure management tasks to cloud providers, allowing developers to focus on code and business logic.
  • Serverless computing runs code only on-demand on a per-request basis, scaling transparently with the number of requests being served.
  • End users only pay for resources being used, never for idle capacity.
  • Code is executed as individual functions inside a stateless container, with no prior execution context required for each request.
  • Applications that qualify for a serverless architecture include short-running stateless functions, seasonal workloads with varying off-peak and peaks, production volumetric data with too much idle time, event-based processing, asynchronous request processing, and microservices that can be built as functions that are stateless.
  • Serverless is well-suited for use cases around data and event processing, IoT, microservices, and mobile backends.
  • Challenges worth considering include vendor dependency and potential lock-in, long-running processes that may be better suited for traditional server environments, and delays associated with starting up a new instance of a function to serve a new request.

Cloud Native applications:

  • A cloud native application is an application developed specifically for the cloud environment, or an existing app that has been refactored and reconfigured with cloud native principles.
  • A cloud native application consists of microservices that work together to comprise an application, yet each can be independently scaled and iterated through automation and orchestration processes.
  • Microservices are often packaged in containers, which are executable units of software that can be run anywhere, enabling frequent, iterative improvement of cloud native applications without disrupting the end-user experience.
  • Cloud native applications are unlike traditional, or monolithic applications, which are built out of one huge piece of software that tightly couples the user interface, business-logic layer, and data-layer.
  • Cloud native apps can be used for anything that lives in the cloud and should be designed with standardized logging, events, and distributed tracing to enable innovation and business agility.
  • Cloud native apps are about enterprise and engineering at scale, and the benefits include enabling innovation, business agility, and commoditization of the solution stack.

DevOps:

  • DevOps is a collaborative approach where development, operations, and quality assurance teams work together to continuously deliver software with the goal of improving productivity, reducing overhead, and accelerating customer feedback cycles.
  • DevOps applies agile and lean thinking principles to all stakeholders involved in the software supply chain, including customers, suppliers, and partners.
  • The DevOps process involves Continuous Delivery, Continuous Integration, Continuous Deployment, Continuous Monitoring, and Delivery Pipeline.
  • DevOps best practices are especially valuable for cloud-native applications due to their complex distributed system and rapid release cycles.
  • DevOps provides capabilities such as programmatically provisioning servers, building middleware, automating installation processes, creating fully automated deployment pipelines, and defining how people work together to build, deploy, and manage applications in a cloud-native approach.
  • DevOps also enables low-cost, production-like test environments and enables systems to be rebuilt quickly and reliably in the event of compromise or natural disasters.

Application modernization:

  • Many organizations have invested in existing applications that are siloed in legacy systems and are difficult and expensive to update and maintain.
  • Application modernization can unlock benefits such as accelerating digital transformations, enabling organizations to take advantage of new technologies and services, and becoming more responsive to customers' needs and changing market dynamics.
  • Cloud computing is one of the three main ingredients in application modernization.
  • The three main transformations in application modernization are changes in architecture, infrastructure, and ways of working (how organizations deliver).
  • Traditional applications were monolithic and ran on physical servers using waterfall style development.
  • Most organizations today use a distributed architecture with web services talking to each other, databases on the backend, and front-ends that go through all of that.
  • Infrastructure has evolved to virtualization and more density, and agile development is the norm.
  • The next phase of application modernization involves a microservice architecture with small, focused services, rest-based communication, and more independence.
  • Cloud infrastructure, both public and private, is a key component of modernization.
  • DevOps and site reliability engineering are critical to modernization in how applications are delivered, built, and what they are.
  • Microservices and cloud infrastructure are tied, and organizations need these new ways of working to take advantage of these new infrastructures and architectures.
  • Speed, resiliency, and adaptability are critical components of modernization. Organizations cannot rely on one-year project plans and need to be more agile.


Module 5:

Cloud security:

  • Cloud computing has a shared responsibility model, where the cloud provider is responsible for managing the security of the platform, while the user is responsible for securing the applications, workload, and data.
  • The adoption model of cloud computing (IaaS, PaaS, SaaS) impacts the level of control and responsibility the user has.
  • When designing a secure architecture for data, encryption at rest and in motion, key management, and access control are crucial.
  • Application security should be ensured through dynamic and static scanning of code and images.
  • Identity and access management are important to control who can access the application and data.
  • Network access control and protection against cyber-attacks should also be considered.
  • Continuous security monitoring is necessary to ensure compliance and manage threats.
  • A set of tools and approach is required to manage security and compliance posture.
  • Insights about posture, compliance, and threats can be gained from the deployment environment, such as security events, audit logs, and flow logs.
  • Actionable intelligence is needed to remediate any vulnerabilities or threats identified.
  • DevOps should be approached with security as a forethought, not an afterthought.
  • SecDevOps approach should be used to embed security into the entire lifecycle of building, managing, and running applications.
  • Security should be shifted left and incorporated into the planning and design phases of the process.
  • A secure design and architecture should be developed, taking into account the data classification, applications, and integrations required.
  • Security should be embedded into the building process of applications, including the use of security-aware applications and encryption of sensitive data.
  • A closed loop process should be established to remediate any issues found and adapt to the evolving threats landscape.

Identity and Access management:

  • Data loss and leakage is the top cloud security concern of cybersecurity professionals.
  • Identity and Access Management (IAM) works as the first line of defense to authenticate and authorize users, and provide user-specific access to cloud resources, services, and applications.
  • IAM has three main types of users: Administrative Users, Developer Users, and Application Users.
  • The key components of IAM are Authentication, Multifactor authentication, Cloud Directory services, Reporting, Audit and Compliance, and User and Service Access Management.
  • Cloud providers offer IAM services, including the ability to create access groups, add users to access groups, and manage access for existing users.
  • Access groups provide a more streamlined access assignment process as compared to assigning individual access to each user and help reduce the number of policies in an account.
  • IAM controls help secure sensitive accounts of administrators or developers by provisioning users, setting password policies, using multifactor authentication, and immediately de-provisioning access when users leave or change roles.


cloud encryption:

  • Encryption is a crucial part of data security and privacy in public cloud environments.
  • Encryption ensures that only authorized users have access to sensitive data and makes data unreadable and meaningless when accessed without authorization.
  • Cloud providers offer various cloud encryption services such as limited encryption of sensitive data or end-to-end encryption of all data uploaded to the cloud.
  • Encryption needs to protect data in three states: at rest, in transit, and in use.
  • Keys need to be managed securely, and losing them can result in data becoming unreadable.
  • Cloud providers offer multi-cloud data encryption services that combine to deliver scalability and flexibility to protect sensitive workloads across the enterprise.
  • Encryption does not eliminate data security risk but separates the security risk from the data itself by moving security to the encryption keys.
  • Key Management Services offered by some cloud providers help perform life cycle management for encryption keys and enable customers to encrypt sensitive data at rest and manage the entire lifecycle of cryptographic keys.
  • Best practices for encryption key management include storing keys separately from the encrypted data, taking key backups offsite, refreshing keys periodically, and implementing multi-factor authentication for both the master and recovery keys.

cloud monitoring:

  • Cloud-based deployments can be complex and require monitoring across the entire stack of applications and services.
  • Cloud monitoring solutions assess data, application, and infrastructure behaviors for performance, resource allocation, network availability, compliance, and security risks and threats.
  • Cloud monitoring is not just about automated tools, it includes strategies, practices, and processes for analyzing, tracking, and managing cloud-based services and applications.
  • Cloud monitoring helps accelerate the diagnosis and resolution of performance incidents, control the cost of monitoring infrastructure, and mitigate the impact of abnormal situations with proactive notifications.
  • Cloud monitoring solutions provide visibility and control over an organization's entire cloud-based infrastructure, including real-time monitoring of virtual machines, services, databases, and applications.
  • Infrastructure, Database, and Application Performance monitoring are categories of cloud monitoring tools.
  • Cloud monitoring best practices include leveraging end-user experience monitoring solutions, moving all infrastructure under one unified monitoring platform, tracking cloud resource usage and cost, increasing cloud monitoring automation, and simulating outages and breach scenarios.
  • Cloud monitoring is essential for organizations looking to optimize cloud resources for cost and performance and create better customer experiences.

Trending Discussions on Cloud

How to solve Invalid credentials problem in bitbucket?

Bitbucket Cloud recently stopped supporting account passwords for Git authentication

Spring Cloud Gateway; Spring MVC found on classpath, which is incompatible with Spring Cloud Gateway Issue

how to specify to not allow any data backup with android:dataExtractionRules and

Can't deploy Cloud Functions because of "Unhandled error cleaning up build images"

App Engine Python 2.7 - ImportError: cannot import name apiproxy

Google app engine deployment fails- Error while finding module specification for 'pip' (AttributeError: module '__main__' has no attribute '__file__')

Netlify says, "error Gatsby requires Node.js 14.15.0 or higher (you have v12.18.0)"β€”yet I have the newest Node version?

App Store Connect Upload Error "You haven't been given access to cloud-managed distribution certificates"

Python 3.10 pattern matching (PEP 634) - wildcard in string

QUESTION

How to solve Invalid credentials problem in bitbucket?

Asked 2022-Mar-31 at 13:55

My bitbucket password is correct because I can easily login with this password. When I try to push a project or file to bitbucket it shows Invalid credentials error.

1$ git push -u origin master
2fatal: Invalid credentials
3Password for 'https://username@bitbucket.org':
4remote: Bitbucket Cloud recently stopped supporting account passwords for Git authentication.
5remote: See our community post for more details: https://atlassian.community/t5/x/x/ba-p/1948231
6remote: App passwords are recommended for most use cases and can be created in your Personal settings:
7remote: https://bitbucket.org/account/settings/app-passwords/
8fatal: Authentication failed for 'https://bitbucket.org/username/demo.git/'
9

How can I solve this issue?

ANSWER

Answered 2022-Mar-07 at 11:47

Go to credential Manager -> Windown Credential -> Add a generic credential fill up the fields Network address: git:https://bitbucket.org Username: App Password: Solver from Here

Source https://stackoverflow.com/questions/71378839

QUESTION

Bitbucket Cloud recently stopped supporting account passwords for Git authentication

Asked 2022-Mar-16 at 20:19

I have pushed the code in the morning it was working fine but now I'm trying to push code but getting this error: Bitbucket Cloud recently stopped supporting account passwords for Git authentication

ANSWER

Answered 2022-Mar-15 at 05:06

I've also faced this issue..

  1. Then I opened my Bitbucket account. 2)Then at the bottom left corner you will find an icon with your username's initials. 3)Click on that and go to settings and click on Personal Settings 4)In personal Settings, in the left panel you will find "access management"
  2. Under that you will find "app passwords"
  3. Click on that --> Then you will routed to create app password page
  4. There you can give the required permissions and create password (Note: You have to store that password somewhere as it will be displayed only once) 8)When you push the code to any repo...use your app password instead of your account password (No need to change any settings in the git_config file)

Happy programming :) !

Source https://stackoverflow.com/questions/71472009

QUESTION

Spring Cloud Gateway; Spring MVC found on classpath, which is incompatible with Spring Cloud Gateway Issue

Asked 2022-Mar-16 at 07:16

I got this below error when run the API-GATEWAY, I tried so many ways but I couldn't solve this issue.

Description:

Spring MVC found on classpath, which is incompatible with Spring Cloud Gateway.

Action:

Please set spring.main.web-application-type=reactive or remove spring-boot-starter-web dependency.

Main Class

1package com.sample.apigateway;
2
3import org.springframework.boot.SpringApplication;
4import org.springframework.boot.autoconfigure.SpringBootApplication;
5import org.springframework.cloud.netflix.eureka.EnableEurekaClient;
6
7@SpringBootApplication
8@EnableEurekaClient
9public class ApiGatewayApplication {
10
11    public static void main(String[] args) {
12        SpringApplication.run(ApiGatewayApplication.class, args);
13    }
14
15}
16

application.yml

1package com.sample.apigateway;
2
3import org.springframework.boot.SpringApplication;
4import org.springframework.boot.autoconfigure.SpringBootApplication;
5import org.springframework.cloud.netflix.eureka.EnableEurekaClient;
6
7@SpringBootApplication
8@EnableEurekaClient
9public class ApiGatewayApplication {
10
11    public static void main(String[] args) {
12        SpringApplication.run(ApiGatewayApplication.class, args);
13    }
14
15}
16spring:
17  application:
18    name: GATEWAY-SERVICE
19
20  cloud:
21    gateway:
22      routes:
23        - id: USER-SERVICE
24          uri: lb://USER-SERVICE
25          predicates:
26            - Path=/users/**
27        - id: DEPARTMENT-SERVICE
28          uri: lb://DEPARTMENT-SERVICE
29          predicates:
30            - Path=/departments/**
31
32eureka:
33  client:
34    register-with-eureka: true
35    fetch-registry: true
36    service-url:
37      defaultZone: http://localhost:8761/eureka/
38
39  instance:
40    hostname: localhost
41
42server:
43  port: 9191
44

pom.xml

1package com.sample.apigateway;
2
3import org.springframework.boot.SpringApplication;
4import org.springframework.boot.autoconfigure.SpringBootApplication;
5import org.springframework.cloud.netflix.eureka.EnableEurekaClient;
6
7@SpringBootApplication
8@EnableEurekaClient
9public class ApiGatewayApplication {
10
11    public static void main(String[] args) {
12        SpringApplication.run(ApiGatewayApplication.class, args);
13    }
14
15}
16spring:
17  application:
18    name: GATEWAY-SERVICE
19
20  cloud:
21    gateway:
22      routes:
23        - id: USER-SERVICE
24          uri: lb://USER-SERVICE
25          predicates:
26            - Path=/users/**
27        - id: DEPARTMENT-SERVICE
28          uri: lb://DEPARTMENT-SERVICE
29          predicates:
30            - Path=/departments/**
31
32eureka:
33  client:
34    register-with-eureka: true
35    fetch-registry: true
36    service-url:
37      defaultZone: http://localhost:8761/eureka/
38
39  instance:
40    hostname: localhost
41
42server:
43  port: 9191
44<?xml version="1.0" encoding="UTF-8"?>
45<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
46         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd">
47    <modelVersion>4.0.0</modelVersion>
48    <parent>
49        <groupId>org.springframework.boot</groupId>
50        <artifactId>spring-boot-starter-parent</artifactId>
51        <version>2.5.3</version>
52        <relativePath/> <!-- lookup parent from repository -->
53    </parent>
54    <groupId>com.sample.apigateway</groupId>
55    <artifactId>apigateway</artifactId>
56    <version>0.0.1-SNAPSHOT</version>
57    <name>apigateway</name>
58    <description>Demo project for Spring Boot</description>
59    <properties>
60        <java.version>11</java.version>
61        <spring-cloud.version>2020.0.3</spring-cloud.version>
62    </properties>
63    <dependencies>
64        <dependency>
65            <groupId>org.springframework.boot</groupId>
66            <artifactId>spring-boot-starter-actuator</artifactId>
67        </dependency>
68        <dependency>
69            <groupId>org.springframework.boot</groupId>
70            <artifactId>spring-boot-starter-webflux</artifactId>
71        </dependency>
72        <dependency>
73            <groupId>io.springfox</groupId>
74            <artifactId>springfox-boot-starter</artifactId>
75            <version>3.0.0</version>
76        </dependency>
77        <dependency>
78            <groupId>org.springframework.cloud</groupId>
79            <artifactId>spring-cloud-starter-gateway</artifactId>
80        </dependency>
81        <dependency>
82            <groupId>org.springframework.cloud</groupId>
83            <artifactId>spring-cloud-starter-netflix-eureka-client</artifactId>
84        </dependency>
85        <dependency>
86            <groupId>org.springframework.cloud</groupId>
87            <artifactId>spring-cloud-starter-netflix-eureka-server</artifactId>
88        </dependency>
89
90        <dependency>
91            <groupId>org.springframework.boot</groupId>
92            <artifactId>spring-boot-starter-test</artifactId>
93            <scope>test</scope>
94        </dependency>
95    </dependencies>
96    <dependencyManagement>
97        <dependencies>
98            <dependency>
99                <groupId>org.springframework.cloud</groupId>
100                <artifactId>spring-cloud-dependencies</artifactId>
101                <version>${spring-cloud.version}</version>
102                <type>pom</type>
103                <scope>import</scope>
104            </dependency>
105        </dependencies>
106    </dependencyManagement>
107
108    <build>
109        <plugins>
110            <plugin>
111                <groupId>org.springframework.boot</groupId>
112                <artifactId>spring-boot-maven-plugin</artifactId>
113            </plugin>
114        </plugins>
115    </build>
116
117</project>
118

Full Error Message

1package com.sample.apigateway;
2
3import org.springframework.boot.SpringApplication;
4import org.springframework.boot.autoconfigure.SpringBootApplication;
5import org.springframework.cloud.netflix.eureka.EnableEurekaClient;
6
7@SpringBootApplication
8@EnableEurekaClient
9public class ApiGatewayApplication {
10
11    public static void main(String[] args) {
12        SpringApplication.run(ApiGatewayApplication.class, args);
13    }
14
15}
16spring:
17  application:
18    name: GATEWAY-SERVICE
19
20  cloud:
21    gateway:
22      routes:
23        - id: USER-SERVICE
24          uri: lb://USER-SERVICE
25          predicates:
26            - Path=/users/**
27        - id: DEPARTMENT-SERVICE
28          uri: lb://DEPARTMENT-SERVICE
29          predicates:
30            - Path=/departments/**
31
32eureka:
33  client:
34    register-with-eureka: true
35    fetch-registry: true
36    service-url:
37      defaultZone: http://localhost:8761/eureka/
38
39  instance:
40    hostname: localhost
41
42server:
43  port: 9191
44<?xml version="1.0" encoding="UTF-8"?>
45<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
46         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd">
47    <modelVersion>4.0.0</modelVersion>
48    <parent>
49        <groupId>org.springframework.boot</groupId>
50        <artifactId>spring-boot-starter-parent</artifactId>
51        <version>2.5.3</version>
52        <relativePath/> <!-- lookup parent from repository -->
53    </parent>
54    <groupId>com.sample.apigateway</groupId>
55    <artifactId>apigateway</artifactId>
56    <version>0.0.1-SNAPSHOT</version>
57    <name>apigateway</name>
58    <description>Demo project for Spring Boot</description>
59    <properties>
60        <java.version>11</java.version>
61        <spring-cloud.version>2020.0.3</spring-cloud.version>
62    </properties>
63    <dependencies>
64        <dependency>
65            <groupId>org.springframework.boot</groupId>
66            <artifactId>spring-boot-starter-actuator</artifactId>
67        </dependency>
68        <dependency>
69            <groupId>org.springframework.boot</groupId>
70            <artifactId>spring-boot-starter-webflux</artifactId>
71        </dependency>
72        <dependency>
73            <groupId>io.springfox</groupId>
74            <artifactId>springfox-boot-starter</artifactId>
75            <version>3.0.0</version>
76        </dependency>
77        <dependency>
78            <groupId>org.springframework.cloud</groupId>
79            <artifactId>spring-cloud-starter-gateway</artifactId>
80        </dependency>
81        <dependency>
82            <groupId>org.springframework.cloud</groupId>
83            <artifactId>spring-cloud-starter-netflix-eureka-client</artifactId>
84        </dependency>
85        <dependency>
86            <groupId>org.springframework.cloud</groupId>
87            <artifactId>spring-cloud-starter-netflix-eureka-server</artifactId>
88        </dependency>
89
90        <dependency>
91            <groupId>org.springframework.boot</groupId>
92            <artifactId>spring-boot-starter-test</artifactId>
93            <scope>test</scope>
94        </dependency>
95    </dependencies>
96    <dependencyManagement>
97        <dependencies>
98            <dependency>
99                <groupId>org.springframework.cloud</groupId>
100                <artifactId>spring-cloud-dependencies</artifactId>
101                <version>${spring-cloud.version}</version>
102                <type>pom</type>
103                <scope>import</scope>
104            </dependency>
105        </dependencies>
106    </dependencyManagement>
107
108    <build>
109        <plugins>
110            <plugin>
111                <groupId>org.springframework.boot</groupId>
112                <artifactId>spring-boot-maven-plugin</artifactId>
113            </plugin>
114        </plugins>
115    </build>
116
117</project>
1182021-07-30 23:12:02.420  WARN 19032 --- [           main] ConfigServletWebServerApplicationContext : Exception encountered during context initialization - cancelling refresh attempt: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'org.springframework.cloud.gateway.config.GatewayClassPathWarningAutoConfiguration$SpringMvcFoundOnClasspathConfiguration': Instantiation of bean failed; nested exception is org.springframework.beans.BeanInstantiationException: Failed to instantiate [org.springframework.cloud.gateway.config.GatewayClassPathWarningAutoConfiguration$SpringMvcFoundOnClasspathConfiguration]: Constructor threw exception; nested exception is org.springframework.cloud.gateway.support.MvcFoundOnClasspathException
1192021-07-30 23:12:02.423  INFO 19032 --- [           main] o.apache.catalina.core.StandardService   : Stopping service [Tomcat]
1202021-07-30 23:12:02.441  INFO 19032 --- [           main] ConditionEvaluationReportLoggingListener : 
121
122Error starting ApplicationContext. To display the conditions report re-run your application with 'debug' enabled.
1232021-07-30 23:12:02.454 ERROR 19032 --- [           main] o.s.b.d.LoggingFailureAnalysisReporter   : 
124
125***************************
126APPLICATION FAILED TO START
127***************************
128
129Description:
130
131Spring MVC found on classpath, which is incompatible with Spring Cloud Gateway.
132
133Action:
134
135Please set spring.main.web-application-type=reactive or remove spring-boot-starter-web dependency.
136
137
138Process finished with exit code 1
139

ANSWER

Answered 2021-Aug-01 at 06:17

Please note that Spring Cloud Gateway is not compatible with Spring MVC (spring-boot-starter-web). This is outlined in section "How to include Spring Cloud Gateway in the official reference documentation":

Spring Cloud Gateway is built on Spring Boot 2.x, Spring WebFlux, and Project Reactor. As a consequence, many of the familiar synchronous libraries (Spring Data and Spring Security, for example) and patterns you know may not apply when you use Spring Cloud Gateway.

Additionally, it is stated that:

Spring Cloud Gateway requires the Netty runtime provided by Spring Boot and Spring Webflux. It does not work in a traditional Servlet Container or when built as a WAR.

As already suggested by the error message, you would need to remove the dependency on spring-boot-starter-web. You can list all your direct and transitive dependencies with the following command:

1package com.sample.apigateway;
2
3import org.springframework.boot.SpringApplication;
4import org.springframework.boot.autoconfigure.SpringBootApplication;
5import org.springframework.cloud.netflix.eureka.EnableEurekaClient;
6
7@SpringBootApplication
8@EnableEurekaClient
9public class ApiGatewayApplication {
10
11    public static void main(String[] args) {
12        SpringApplication.run(ApiGatewayApplication.class, args);
13    }
14
15}
16spring:
17  application:
18    name: GATEWAY-SERVICE
19
20  cloud:
21    gateway:
22      routes:
23        - id: USER-SERVICE
24          uri: lb://USER-SERVICE
25          predicates:
26            - Path=/users/**
27        - id: DEPARTMENT-SERVICE
28          uri: lb://DEPARTMENT-SERVICE
29          predicates:
30            - Path=/departments/**
31
32eureka:
33  client:
34    register-with-eureka: true
35    fetch-registry: true
36    service-url:
37      defaultZone: http://localhost:8761/eureka/
38
39  instance:
40    hostname: localhost
41
42server:
43  port: 9191
44<?xml version="1.0" encoding="UTF-8"?>
45<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
46         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd">
47    <modelVersion>4.0.0</modelVersion>
48    <parent>
49        <groupId>org.springframework.boot</groupId>
50        <artifactId>spring-boot-starter-parent</artifactId>
51        <version>2.5.3</version>
52        <relativePath/> <!-- lookup parent from repository -->
53    </parent>
54    <groupId>com.sample.apigateway</groupId>
55    <artifactId>apigateway</artifactId>
56    <version>0.0.1-SNAPSHOT</version>
57    <name>apigateway</name>
58    <description>Demo project for Spring Boot</description>
59    <properties>
60        <java.version>11</java.version>
61        <spring-cloud.version>2020.0.3</spring-cloud.version>
62    </properties>
63    <dependencies>
64        <dependency>
65            <groupId>org.springframework.boot</groupId>
66            <artifactId>spring-boot-starter-actuator</artifactId>
67        </dependency>
68        <dependency>
69            <groupId>org.springframework.boot</groupId>
70            <artifactId>spring-boot-starter-webflux</artifactId>
71        </dependency>
72        <dependency>
73            <groupId>io.springfox</groupId>
74            <artifactId>springfox-boot-starter</artifactId>
75            <version>3.0.0</version>
76        </dependency>
77        <dependency>
78            <groupId>org.springframework.cloud</groupId>
79            <artifactId>spring-cloud-starter-gateway</artifactId>
80        </dependency>
81        <dependency>
82            <groupId>org.springframework.cloud</groupId>
83            <artifactId>spring-cloud-starter-netflix-eureka-client</artifactId>
84        </dependency>
85        <dependency>
86            <groupId>org.springframework.cloud</groupId>
87            <artifactId>spring-cloud-starter-netflix-eureka-server</artifactId>
88        </dependency>
89
90        <dependency>
91            <groupId>org.springframework.boot</groupId>
92            <artifactId>spring-boot-starter-test</artifactId>
93            <scope>test</scope>
94        </dependency>
95    </dependencies>
96    <dependencyManagement>
97        <dependencies>
98            <dependency>
99                <groupId>org.springframework.cloud</groupId>
100                <artifactId>spring-cloud-dependencies</artifactId>
101                <version>${spring-cloud.version}</version>
102                <type>pom</type>
103                <scope>import</scope>
104            </dependency>
105        </dependencies>
106    </dependencyManagement>
107
108    <build>
109        <plugins>
110            <plugin>
111                <groupId>org.springframework.boot</groupId>
112                <artifactId>spring-boot-maven-plugin</artifactId>
113            </plugin>
114        </plugins>
115    </build>
116
117</project>
1182021-07-30 23:12:02.420  WARN 19032 --- [           main] ConfigServletWebServerApplicationContext : Exception encountered during context initialization - cancelling refresh attempt: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'org.springframework.cloud.gateway.config.GatewayClassPathWarningAutoConfiguration$SpringMvcFoundOnClasspathConfiguration': Instantiation of bean failed; nested exception is org.springframework.beans.BeanInstantiationException: Failed to instantiate [org.springframework.cloud.gateway.config.GatewayClassPathWarningAutoConfiguration$SpringMvcFoundOnClasspathConfiguration]: Constructor threw exception; nested exception is org.springframework.cloud.gateway.support.MvcFoundOnClasspathException
1192021-07-30 23:12:02.423  INFO 19032 --- [           main] o.apache.catalina.core.StandardService   : Stopping service [Tomcat]
1202021-07-30 23:12:02.441  INFO 19032 --- [           main] ConditionEvaluationReportLoggingListener : 
121
122Error starting ApplicationContext. To display the conditions report re-run your application with 'debug' enabled.
1232021-07-30 23:12:02.454 ERROR 19032 --- [           main] o.s.b.d.LoggingFailureAnalysisReporter   : 
124
125***************************
126APPLICATION FAILED TO START
127***************************
128
129Description:
130
131Spring MVC found on classpath, which is incompatible with Spring Cloud Gateway.
132
133Action:
134
135Please set spring.main.web-application-type=reactive or remove spring-boot-starter-web dependency.
136
137
138Process finished with exit code 1
139mvn dependency:tree
140

This will reveal that spring-cloud-starter-netflix-eureka-server depends on spring-boot-starter-web. Once you remove the following lines from your pom.xml, your application should start up:

1package com.sample.apigateway;
2
3import org.springframework.boot.SpringApplication;
4import org.springframework.boot.autoconfigure.SpringBootApplication;
5import org.springframework.cloud.netflix.eureka.EnableEurekaClient;
6
7@SpringBootApplication
8@EnableEurekaClient
9public class ApiGatewayApplication {
10
11    public static void main(String[] args) {
12        SpringApplication.run(ApiGatewayApplication.class, args);
13    }
14
15}
16spring:
17  application:
18    name: GATEWAY-SERVICE
19
20  cloud:
21    gateway:
22      routes:
23        - id: USER-SERVICE
24          uri: lb://USER-SERVICE
25          predicates:
26            - Path=/users/**
27        - id: DEPARTMENT-SERVICE
28          uri: lb://DEPARTMENT-SERVICE
29          predicates:
30            - Path=/departments/**
31
32eureka:
33  client:
34    register-with-eureka: true
35    fetch-registry: true
36    service-url:
37      defaultZone: http://localhost:8761/eureka/
38
39  instance:
40    hostname: localhost
41
42server:
43  port: 9191
44<?xml version="1.0" encoding="UTF-8"?>
45<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
46         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd">
47    <modelVersion>4.0.0</modelVersion>
48    <parent>
49        <groupId>org.springframework.boot</groupId>
50        <artifactId>spring-boot-starter-parent</artifactId>
51        <version>2.5.3</version>
52        <relativePath/> <!-- lookup parent from repository -->
53    </parent>
54    <groupId>com.sample.apigateway</groupId>
55    <artifactId>apigateway</artifactId>
56    <version>0.0.1-SNAPSHOT</version>
57    <name>apigateway</name>
58    <description>Demo project for Spring Boot</description>
59    <properties>
60        <java.version>11</java.version>
61        <spring-cloud.version>2020.0.3</spring-cloud.version>
62    </properties>
63    <dependencies>
64        <dependency>
65            <groupId>org.springframework.boot</groupId>
66            <artifactId>spring-boot-starter-actuator</artifactId>
67        </dependency>
68        <dependency>
69            <groupId>org.springframework.boot</groupId>
70            <artifactId>spring-boot-starter-webflux</artifactId>
71        </dependency>
72        <dependency>
73            <groupId>io.springfox</groupId>
74            <artifactId>springfox-boot-starter</artifactId>
75            <version>3.0.0</version>
76        </dependency>
77        <dependency>
78            <groupId>org.springframework.cloud</groupId>
79            <artifactId>spring-cloud-starter-gateway</artifactId>
80        </dependency>
81        <dependency>
82            <groupId>org.springframework.cloud</groupId>
83            <artifactId>spring-cloud-starter-netflix-eureka-client</artifactId>
84        </dependency>
85        <dependency>
86            <groupId>org.springframework.cloud</groupId>
87            <artifactId>spring-cloud-starter-netflix-eureka-server</artifactId>
88        </dependency>
89
90        <dependency>
91            <groupId>org.springframework.boot</groupId>
92            <artifactId>spring-boot-starter-test</artifactId>
93            <scope>test</scope>
94        </dependency>
95    </dependencies>
96    <dependencyManagement>
97        <dependencies>
98            <dependency>
99                <groupId>org.springframework.cloud</groupId>
100                <artifactId>spring-cloud-dependencies</artifactId>
101                <version>${spring-cloud.version}</version>
102                <type>pom</type>
103                <scope>import</scope>
104            </dependency>
105        </dependencies>
106    </dependencyManagement>
107
108    <build>
109        <plugins>
110            <plugin>
111                <groupId>org.springframework.boot</groupId>
112                <artifactId>spring-boot-maven-plugin</artifactId>
113            </plugin>
114        </plugins>
115    </build>
116
117</project>
1182021-07-30 23:12:02.420  WARN 19032 --- [           main] ConfigServletWebServerApplicationContext : Exception encountered during context initialization - cancelling refresh attempt: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'org.springframework.cloud.gateway.config.GatewayClassPathWarningAutoConfiguration$SpringMvcFoundOnClasspathConfiguration': Instantiation of bean failed; nested exception is org.springframework.beans.BeanInstantiationException: Failed to instantiate [org.springframework.cloud.gateway.config.GatewayClassPathWarningAutoConfiguration$SpringMvcFoundOnClasspathConfiguration]: Constructor threw exception; nested exception is org.springframework.cloud.gateway.support.MvcFoundOnClasspathException
1192021-07-30 23:12:02.423  INFO 19032 --- [           main] o.apache.catalina.core.StandardService   : Stopping service [Tomcat]
1202021-07-30 23:12:02.441  INFO 19032 --- [           main] ConditionEvaluationReportLoggingListener : 
121
122Error starting ApplicationContext. To display the conditions report re-run your application with 'debug' enabled.
1232021-07-30 23:12:02.454 ERROR 19032 --- [           main] o.s.b.d.LoggingFailureAnalysisReporter   : 
124
125***************************
126APPLICATION FAILED TO START
127***************************
128
129Description:
130
131Spring MVC found on classpath, which is incompatible with Spring Cloud Gateway.
132
133Action:
134
135Please set spring.main.web-application-type=reactive or remove spring-boot-starter-web dependency.
136
137
138Process finished with exit code 1
139mvn dependency:tree
140<dependency>
141    <groupId>org.springframework.cloud</groupId>
142    <artifactId>spring-cloud-starter-netflix-eureka-server</artifactId>
143</dependency>
144

Source https://stackoverflow.com/questions/68587832

QUESTION

how to specify to not allow any data backup with android:dataExtractionRules and

Asked 2022-Mar-01 at 12:45

My current android application targets 12 and higher.

I do not want to allow backup of any type and currently have these manifest settings

1    android:allowBackup="false"
2    android:fullBackupContent="false"
3    
4

however the android:allowBackup="false" setting gives the following warning now

1    android:allowBackup="false"
2    android:fullBackupContent="false"
3    
4The attribute android:allowBackup is deprecated from Android 12 and higher and may be removed in future versions. Consider adding the attribute android:dataExtractionRules specifying an @xml resource which configures cloud backups and device transfers on Android 12 and higher.
5

Ive looked at the examples for android:dataExtractionRules xml and none of them show how to configure the equivalent of allowBackup="false".

what am i missing?

is it possible to achieve allowBackup="false" with the use of android:dataExtractionRules xml

ANSWER

Answered 2022-Feb-10 at 15:28

It's usually better to disable backups only for debug builds:

1    android:allowBackup="false"
2    android:fullBackupContent="false"
3    
4The attribute android:allowBackup is deprecated from Android 12 and higher and may be removed in future versions. Consider adding the attribute android:dataExtractionRules specifying an @xml resource which configures cloud backups and device transfers on Android 12 and higher.
5<application
6    android:fullBackupContent="false"
7    tools:replace="android:fullBackupContent"
8    tools:ignore="DataExtractionRules,MissingApplicationIcon"/>
9

One can as well merge an alternate src/debug/res/xml/data_extraction_rules.xml.

Source https://stackoverflow.com/questions/70365809

QUESTION

Can't deploy Cloud Functions because of "Unhandled error cleaning up build images"

Asked 2022-Feb-10 at 19:11

I've deployed hundreds of function and this is the first time I encounter this issue. Simply, it stops deploying function process, saying:

Unhandled error cleaning up build images. This could result in a small monthly bill if not corrected. You can attempt to delete these images by redeploying or you can delete them manually at https://console.cloud.google.com/gcr/images/[project-name]/us/gcf

The way I deploy is through Firebase CLI command: firebase deploy --only functions:nameOfFunction

Question is what are those images I have to delete? Why? How can I solve it?

ANSWER

Answered 2021-Aug-01 at 15:56

Cloud Functions uses another product called Cloud Build to build the server images that actually get deployed. Those images are stored in Cloud Storage, and that storage is billed to your account.

Read more about it:

Watch:

You should be able to locate and delete the files manually in the Google Cloud console. But it sounds like there is a bug here with the files not being cleaned up automatically, so you contact Firebase support directly.

Source https://stackoverflow.com/questions/68611817

QUESTION

App Engine Python 2.7 - ImportError: cannot import name apiproxy

Asked 2022-Feb-08 at 08:52

With the upgrade to Google Cloud SDK 360.0.0-0 i started seeing the following error when running the dev_appserver.py command for my Python 2.7 App Engine project.

1  File "/home/..................py", line 6, in <module>
2    from google.appengine.ext import ndb
3  File "/usr/lib/google-cloud-sdk/platform/google_appengine/python27/sdk/google/appengine/ext/ndb/__init__.py", line 22, in <module>
4    from tasklets import *
5  File "/usr/lib/google-cloud-sdk/platform/google_appengine/python27/sdk/google/appengine/ext/ndb/tasklets.py", line 85, in <module>
6    from .google_imports import apiproxy_stub_map
7  File "/usr/lib/google-cloud-sdk/platform/google_appengine/python27/sdk/google/appengine/ext/ndb/google_imports.py", line 44, in <module>
8    from google.appengine.runtime import apiproxy as callback
9ImportError: cannot import name apiproxy
10

gcloud version

1  File "/home/..................py", line 6, in <module>
2    from google.appengine.ext import ndb
3  File "/usr/lib/google-cloud-sdk/platform/google_appengine/python27/sdk/google/appengine/ext/ndb/__init__.py", line 22, in <module>
4    from tasklets import *
5  File "/usr/lib/google-cloud-sdk/platform/google_appengine/python27/sdk/google/appengine/ext/ndb/tasklets.py", line 85, in <module>
6    from .google_imports import apiproxy_stub_map
7  File "/usr/lib/google-cloud-sdk/platform/google_appengine/python27/sdk/google/appengine/ext/ndb/google_imports.py", line 44, in <module>
8    from google.appengine.runtime import apiproxy as callback
9ImportError: cannot import name apiproxy
10Google Cloud SDK 360.0.0
11alpha 2021.10.04
12app-engine-python 1.9.95
13app-engine-python-extras 1.9.95
14beta 2021.10.04
15bq 2.0.71
16cloud-build-local 0.5.2
17cloud-datastore-emulator 2.1.0
18core 2021.10.04
19gsutil 5.3
20

ANSWER

Answered 2022-Feb-08 at 08:52
EDIT

This issue seems to have been resolved with Google Cloud SDK version 371


On my debian based system i fixed it by downgrading the app-engine-python component to the previous version

1  File "/home/..................py", line 6, in <module>
2    from google.appengine.ext import ndb
3  File "/usr/lib/google-cloud-sdk/platform/google_appengine/python27/sdk/google/appengine/ext/ndb/__init__.py", line 22, in <module>
4    from tasklets import *
5  File "/usr/lib/google-cloud-sdk/platform/google_appengine/python27/sdk/google/appengine/ext/ndb/tasklets.py", line 85, in <module>
6    from .google_imports import apiproxy_stub_map
7  File "/usr/lib/google-cloud-sdk/platform/google_appengine/python27/sdk/google/appengine/ext/ndb/google_imports.py", line 44, in <module>
8    from google.appengine.runtime import apiproxy as callback
9ImportError: cannot import name apiproxy
10Google Cloud SDK 360.0.0
11alpha 2021.10.04
12app-engine-python 1.9.95
13app-engine-python-extras 1.9.95
14beta 2021.10.04
15bq 2.0.71
16cloud-build-local 0.5.2
17cloud-datastore-emulator 2.1.0
18core 2021.10.04
19gsutil 5.3
20sudo apt-get install google-cloud-sdk-app-engine-python=359.0.0-0
21sudo apt-get install google-cloud-sdk-app-engine-python-extras=359.0.0-0
22

gcloud version (after downgrade)

1  File "/home/..................py", line 6, in <module>
2    from google.appengine.ext import ndb
3  File "/usr/lib/google-cloud-sdk/platform/google_appengine/python27/sdk/google/appengine/ext/ndb/__init__.py", line 22, in <module>
4    from tasklets import *
5  File "/usr/lib/google-cloud-sdk/platform/google_appengine/python27/sdk/google/appengine/ext/ndb/tasklets.py", line 85, in <module>
6    from .google_imports import apiproxy_stub_map
7  File "/usr/lib/google-cloud-sdk/platform/google_appengine/python27/sdk/google/appengine/ext/ndb/google_imports.py", line 44, in <module>
8    from google.appengine.runtime import apiproxy as callback
9ImportError: cannot import name apiproxy
10Google Cloud SDK 360.0.0
11alpha 2021.10.04
12app-engine-python 1.9.95
13app-engine-python-extras 1.9.95
14beta 2021.10.04
15bq 2.0.71
16cloud-build-local 0.5.2
17cloud-datastore-emulator 2.1.0
18core 2021.10.04
19gsutil 5.3
20sudo apt-get install google-cloud-sdk-app-engine-python=359.0.0-0
21sudo apt-get install google-cloud-sdk-app-engine-python-extras=359.0.0-0
22Google Cloud SDK 360.0.0
23alpha 2021.10.04
24app-engine-python 1.9.94
25app-engine-python-extras 1.9.95
26beta 2021.10.04
27bq 2.0.71
28cloud-build-local 0.5.2
29cloud-datastore-emulator 2.1.0
30core 2021.10.04
31gsutil 5.3
32

I created an issue for this bug: https://issuetracker.google.com/issues/202171426

Source https://stackoverflow.com/questions/69465376

QUESTION

Google app engine deployment fails- Error while finding module specification for 'pip' (AttributeError: module '__main__' has no attribute '__file__')

Asked 2022-Jan-08 at 22:02

We are using command prompt c:\gcloud app deploy app.yaml, but get the following error:

1Running "python3 -m pip install --requirement requirements.txt --upgrade --upgrade-strategy only-if-needed --no-warn-script-location --no-warn-conflicts --force-reinstall --no-compile (PIP_CACHE_DIR=/layers/google.python.pip/pipcache PIP_DISABLE_PIP_VERSION_CHECK=1)"
2Step #2 - "build": /layers/google.python.pip/pip/bin/python3: Error while finding module specification for 'pip' (AttributeError: module '__main__' has no attribute '__file__')
3Step #2 - "build": Done "python3 -m pip install --requirement requirements.txt --upgr..." (34.49892ms)
4Step #2 - "build": Failure: (ID: 0ea8a540) /layers/google.python.pip/pip/bin/python3: Error while finding module specification for 'pip' (AttributeError: module '__main__' has no attribute '__file__')
5Step #2 - "build": --------------------------------------------------------------------------------
6Step #2 - "build": Running "mv -f /builder/outputs/output-5577006791947779410 /builder/outputs/output"
7Step #2 - "build": Done "mv -f /builder/outputs/output-5577006791947779410 /builder/o..." (12.758866ms)
8Step #2 - "build": ERROR: failed to build: exit status 1
9Finished Step #2 - "build"
10ERROR
11ERROR: build step 2 "us.gcr.io/gae-runtimes/buildpacks/python37/builder:python37_20211201_3_7_12_RC00" failed: step exited with non-zero status: 145
12

Our Requirements.txt is as below. We are currently on Python 3.7 standard app engine

1Running "python3 -m pip install --requirement requirements.txt --upgrade --upgrade-strategy only-if-needed --no-warn-script-location --no-warn-conflicts --force-reinstall --no-compile (PIP_CACHE_DIR=/layers/google.python.pip/pipcache PIP_DISABLE_PIP_VERSION_CHECK=1)"
2Step #2 - "build": /layers/google.python.pip/pip/bin/python3: Error while finding module specification for 'pip' (AttributeError: module '__main__' has no attribute '__file__')
3Step #2 - "build": Done "python3 -m pip install --requirement requirements.txt --upgr..." (34.49892ms)
4Step #2 - "build": Failure: (ID: 0ea8a540) /layers/google.python.pip/pip/bin/python3: Error while finding module specification for 'pip' (AttributeError: module '__main__' has no attribute '__file__')
5Step #2 - "build": --------------------------------------------------------------------------------
6Step #2 - "build": Running "mv -f /builder/outputs/output-5577006791947779410 /builder/outputs/output"
7Step #2 - "build": Done "mv -f /builder/outputs/output-5577006791947779410 /builder/o..." (12.758866ms)
8Step #2 - "build": ERROR: failed to build: exit status 1
9Finished Step #2 - "build"
10ERROR
11ERROR: build step 2 "us.gcr.io/gae-runtimes/buildpacks/python37/builder:python37_20211201_3_7_12_RC00" failed: step exited with non-zero status: 145
12firebase_admin==3.0.0
13sendgrid==6.9.3
14google-auth==1.35.0
15google-auth-httplib2==0.1.0
16jinja2==3.0.3
17MarkupSafe==2.0.1
18pytz==2021.3
19Flask==2.0.2
20twilio==6.46.0
21httplib2==0.20.2
22requests==2.24.0
23requests_toolbelt==0.9.1
24google-cloud-tasks==2.7.1
25google-cloud-logging==1.15.1
26googleapis-common-protos==1.54.0
27

Please help.The above code was working well before updating the requirements.txt file. We tried to remove gunicorn to allow the system pickup the latest according to documentation here.

We have a subdirectory structure that stores all the .py files in controllers and db definitions in models. Our main.py has the following -

1Running "python3 -m pip install --requirement requirements.txt --upgrade --upgrade-strategy only-if-needed --no-warn-script-location --no-warn-conflicts --force-reinstall --no-compile (PIP_CACHE_DIR=/layers/google.python.pip/pipcache PIP_DISABLE_PIP_VERSION_CHECK=1)"
2Step #2 - "build": /layers/google.python.pip/pip/bin/python3: Error while finding module specification for 'pip' (AttributeError: module '__main__' has no attribute '__file__')
3Step #2 - "build": Done "python3 -m pip install --requirement requirements.txt --upgr..." (34.49892ms)
4Step #2 - "build": Failure: (ID: 0ea8a540) /layers/google.python.pip/pip/bin/python3: Error while finding module specification for 'pip' (AttributeError: module '__main__' has no attribute '__file__')
5Step #2 - "build": --------------------------------------------------------------------------------
6Step #2 - "build": Running "mv -f /builder/outputs/output-5577006791947779410 /builder/outputs/output"
7Step #2 - "build": Done "mv -f /builder/outputs/output-5577006791947779410 /builder/o..." (12.758866ms)
8Step #2 - "build": ERROR: failed to build: exit status 1
9Finished Step #2 - "build"
10ERROR
11ERROR: build step 2 "us.gcr.io/gae-runtimes/buildpacks/python37/builder:python37_20211201_3_7_12_RC00" failed: step exited with non-zero status: 145
12firebase_admin==3.0.0
13sendgrid==6.9.3
14google-auth==1.35.0
15google-auth-httplib2==0.1.0
16jinja2==3.0.3
17MarkupSafe==2.0.1
18pytz==2021.3
19Flask==2.0.2
20twilio==6.46.0
21httplib2==0.20.2
22requests==2.24.0
23requests_toolbelt==0.9.1
24google-cloud-tasks==2.7.1
25google-cloud-logging==1.15.1
26googleapis-common-protos==1.54.0
27sys.path.append(os.path.join(os.path.dirname(__file__), '../controllers'))
28sys.path.append(os.path.join(os.path.dirname(__file__), '../models'))
29

Does anyone know how to debug this error - Error while finding module specification for 'pip' (AttributeError: module '__main__' has no attribute '__file__'). What does this mean?

ANSWER

Answered 2022-Jan-06 at 09:24

Your setuptools version is likely to be yanked:

https://pypi.org/project/setuptools/60.3.0/

Not sure how to fix that without a working pip though.

Source https://stackoverflow.com/questions/70602290

QUESTION

Netlify says, "error Gatsby requires Node.js 14.15.0 or higher (you have v12.18.0)"β€”yet I have the newest Node version?

Asked 2022-Jan-08 at 07:21

After migrating from Remark to MDX, my builds on Netlify are failing.

I get this error when trying to build:

110:13:28 AM: $ npm run build
210:13:29 AM: > blog-gatsby@0.1.0 build /opt/build/repo
310:13:29 AM: > gatsby build
410:13:30 AM: error Gatsby requires Node.js 14.15.0 or higher (you have v12.18.0).
510:13:30 AM: Upgrade Node to the latest stable release: https://gatsby.dev/upgrading-node-js
6

Yet when I run node -v in my terminal, it says v17.2.0.

I assume it's not a coincidence that this happened after migrating. Can the problem be because of my node-modules folder? Or is there something in my gatsby-config.js or package.json files I need to change?

My package.json file:

110:13:28 AM: $ npm run build
210:13:29 AM: > blog-gatsby@0.1.0 build /opt/build/repo
310:13:29 AM: > gatsby build
410:13:30 AM: error Gatsby requires Node.js 14.15.0 or higher (you have v12.18.0).
510:13:30 AM: Upgrade Node to the latest stable release: https://gatsby.dev/upgrading-node-js
6{
7  "name": "blog-gatsby",
8  "private": true,
9  "description": "A starter for a blog powered by Gatsby and Markdown",
10  "version": "0.1.0",
11  "author": "Magnus Kolstad <kolstadmagnus@gmail.com>",
12  "bugs": {
13    "url": "https://kolstadmagnus.no"
14  },
15  "dependencies": {
16    "@mdx-js/mdx": "^1.6.22",
17    "@mdx-js/react": "^1.6.22",
18    "gatsby": "^4.3.0",
19    "gatsby-plugin-feed": "^4.3.0",
20    "gatsby-plugin-gatsby-cloud": "^4.3.0",
21    "gatsby-plugin-google-analytics": "^4.3.0",
22    "gatsby-plugin-image": "^2.3.0",
23    "gatsby-plugin-manifest": "^4.3.0",
24    "gatsby-plugin-mdx": "^3.4.0",
25    "gatsby-plugin-offline": "^5.3.0",
26    "gatsby-plugin-react-helmet": "^5.3.0",
27    "gatsby-plugin-sharp": "^4.3.0",
28    "gatsby-remark-copy-linked-files": "^5.3.0",
29    "gatsby-remark-images": "^6.3.0",
30    "gatsby-remark-prismjs": "^6.3.0",
31    "gatsby-remark-responsive-iframe": "^5.3.0",
32    "gatsby-remark-smartypants": "^5.3.0",
33    "gatsby-source-filesystem": "^4.3.0",
34    "gatsby-transformer-sharp": "^4.3.0",
35    "prismjs": "^1.25.0",
36    "react": "^17.0.1",
37    "react-dom": "^17.0.1",
38    "react-helmet": "^6.1.0",
39    "typeface-merriweather": "0.0.72",
40    "typeface-montserrat": "0.0.75"
41  },
42  "devDependencies": {
43    "prettier": "^2.4.1"
44  },
45  "homepage": "https://kolstadmagnus.no",
46  "keywords": [
47    "blog"
48  ],
49  "license": "0BSD",
50  "main": "n/a",
51  "repository": {
52    "type": "git",
53    "url": "git+https://github.com/gatsbyjs/gatsby-starter-blog.git"
54  },
55  "scripts": {
56    "build": "gatsby build",
57    "develop": "gatsby develop",
58    "format": "prettier --write \"**/*.{js,jsx,ts,tsx,json,md}\"",
59    "start": "gatsby develop",
60    "serve": "gatsby serve",
61    "clean": "gatsby clean",
62    "test": "echo \"Write tests! -> https://gatsby.dev/unit-testing\" && exit 1"
63  }
64}
65

What am I doing wrong here?


Update #1
110:13:28 AM: $ npm run build
210:13:29 AM: > blog-gatsby@0.1.0 build /opt/build/repo
310:13:29 AM: > gatsby build
410:13:30 AM: error Gatsby requires Node.js 14.15.0 or higher (you have v12.18.0).
510:13:30 AM: Upgrade Node to the latest stable release: https://gatsby.dev/upgrading-node-js
6{
7  "name": "blog-gatsby",
8  "private": true,
9  "description": "A starter for a blog powered by Gatsby and Markdown",
10  "version": "0.1.0",
11  "author": "Magnus Kolstad <kolstadmagnus@gmail.com>",
12  "bugs": {
13    "url": "https://kolstadmagnus.no"
14  },
15  "dependencies": {
16    "@mdx-js/mdx": "^1.6.22",
17    "@mdx-js/react": "^1.6.22",
18    "gatsby": "^4.3.0",
19    "gatsby-plugin-feed": "^4.3.0",
20    "gatsby-plugin-gatsby-cloud": "^4.3.0",
21    "gatsby-plugin-google-analytics": "^4.3.0",
22    "gatsby-plugin-image": "^2.3.0",
23    "gatsby-plugin-manifest": "^4.3.0",
24    "gatsby-plugin-mdx": "^3.4.0",
25    "gatsby-plugin-offline": "^5.3.0",
26    "gatsby-plugin-react-helmet": "^5.3.0",
27    "gatsby-plugin-sharp": "^4.3.0",
28    "gatsby-remark-copy-linked-files": "^5.3.0",
29    "gatsby-remark-images": "^6.3.0",
30    "gatsby-remark-prismjs": "^6.3.0",
31    "gatsby-remark-responsive-iframe": "^5.3.0",
32    "gatsby-remark-smartypants": "^5.3.0",
33    "gatsby-source-filesystem": "^4.3.0",
34    "gatsby-transformer-sharp": "^4.3.0",
35    "prismjs": "^1.25.0",
36    "react": "^17.0.1",
37    "react-dom": "^17.0.1",
38    "react-helmet": "^6.1.0",
39    "typeface-merriweather": "0.0.72",
40    "typeface-montserrat": "0.0.75"
41  },
42  "devDependencies": {
43    "prettier": "^2.4.1"
44  },
45  "homepage": "https://kolstadmagnus.no",
46  "keywords": [
47    "blog"
48  ],
49  "license": "0BSD",
50  "main": "n/a",
51  "repository": {
52    "type": "git",
53    "url": "git+https://github.com/gatsbyjs/gatsby-starter-blog.git"
54  },
55  "scripts": {
56    "build": "gatsby build",
57    "develop": "gatsby develop",
58    "format": "prettier --write \"**/*.{js,jsx,ts,tsx,json,md}\"",
59    "start": "gatsby develop",
60    "serve": "gatsby serve",
61    "clean": "gatsby clean",
62    "test": "echo \"Write tests! -> https://gatsby.dev/unit-testing\" && exit 1"
63  }
64}
657:11:59 PM: failed Building production JavaScript and CSS bundles - 20.650s
667:11:59 PM: error Generating JavaScript bundles failed
677:11:59 PM: Module build failed (from ./node_modules/url-loader/dist/cjs.js):
687:11:59 PM: Error: error:0308010C:digital envelope routines::unsupported
697:11:59 PM:     at new Hash (node:internal/crypto/hash:67:19)
707:11:59 PM:     at Object.createHash (node:crypto:130:10)
717:11:59 PM:     at getHashDigest (/opt/build/repo/node_modules/file-loader/node_modules/loader-utils/lib/getHashDigest.js:46:34)
727:11:59 PM:     at /opt/build/repo/node_modules/file-loader/node_modules/loader-utils/lib/interpolateName.js:113:11
737:11:59 PM:     at String.replace (<anonymous>)
747:11:59 PM:     at interpolateName (/opt/build/repo/node_modules/file-loader/node_modules/loader-utils/lib/interpolateName.js:110:8)
757:11:59 PM:     at Object.loader (/opt/build/repo/node_modules/file-loader/dist/index.js:29:48)
767:11:59 PM:     at Object.loader (/opt/build/repo/node_modules/url-loader/dist/index.js:127:19)
777:11:59 PM: ​
787:11:59 PM: ────────────────────────────────────────────────────────────────
797:11:59 PM:   "build.command" failed                                        
807:11:59 PM: ────────────────────────────────────────────────────────────────
817:11:59 PM: ​
827:11:59 PM:   Error message
837:11:59 PM:   Command failed with exit code 1: npm run build
847:11:59 PM: ​
857:11:59 PM:   Error location
867:11:59 PM:   In Build command from Netlify app:
877:11:59 PM:   npm run build
887:11:59 PM: ​
897:11:59 PM:   Resolved config
907:11:59 PM:   build:
917:11:59 PM:     command: npm run build
927:11:59 PM:     commandOrigin: ui
937:11:59 PM:     publish: /opt/build/repo/public
947:11:59 PM:     publishOrigin: ui
957:11:59 PM:   plugins:
967:11:59 PM:     - inputs: {}
977:11:59 PM:       origin: ui
987:11:59 PM:       package: '@netlify/plugin-gatsby'
997:11:59 PM:   redirects:
1007:12:00 PM:     - from: /api/*
101      status: 200
102      to: /.netlify/functions/gatsby
103    - force: true
104      from: https://magnuskolstad.com
105      status: 301
106      to: https://kolstadmagnus.no
107  redirectsOrigin: config
108Caching artifacts
109

ANSWER

Answered 2022-Jan-08 at 07:21

The problem is that you have Node 17.2.0. locally but in Netlify's environment, you are running a lower version (by default it's not set as 17.2.0). So the local environment is OK, Netlify environment is KO because of this mismatch of Node versions.

When Netlify deploys your site it installs and builds again your site so you should ensure that both environments work under the same conditions. Otherwise, both node_modules will differ so your application will have different behavior or eventually won't even build because of dependency errors.

You can easily play with the Node version in multiple ways but I'd recommend using the .nvmrc file. Just run the following command in the root of your project:

110:13:28 AM: $ npm run build
210:13:29 AM: > blog-gatsby@0.1.0 build /opt/build/repo
310:13:29 AM: > gatsby build
410:13:30 AM: error Gatsby requires Node.js 14.15.0 or higher (you have v12.18.0).
510:13:30 AM: Upgrade Node to the latest stable release: https://gatsby.dev/upgrading-node-js
6{
7  "name": "blog-gatsby",
8  "private": true,
9  "description": "A starter for a blog powered by Gatsby and Markdown",
10  "version": "0.1.0",
11  "author": "Magnus Kolstad <kolstadmagnus@gmail.com>",
12  "bugs": {
13    "url": "https://kolstadmagnus.no"
14  },
15  "dependencies": {
16    "@mdx-js/mdx": "^1.6.22",
17    "@mdx-js/react": "^1.6.22",
18    "gatsby": "^4.3.0",
19    "gatsby-plugin-feed": "^4.3.0",
20    "gatsby-plugin-gatsby-cloud": "^4.3.0",
21    "gatsby-plugin-google-analytics": "^4.3.0",
22    "gatsby-plugin-image": "^2.3.0",
23    "gatsby-plugin-manifest": "^4.3.0",
24    "gatsby-plugin-mdx": "^3.4.0",
25    "gatsby-plugin-offline": "^5.3.0",
26    "gatsby-plugin-react-helmet": "^5.3.0",
27    "gatsby-plugin-sharp": "^4.3.0",
28    "gatsby-remark-copy-linked-files": "^5.3.0",
29    "gatsby-remark-images": "^6.3.0",
30    "gatsby-remark-prismjs": "^6.3.0",
31    "gatsby-remark-responsive-iframe": "^5.3.0",
32    "gatsby-remark-smartypants": "^5.3.0",
33    "gatsby-source-filesystem": "^4.3.0",
34    "gatsby-transformer-sharp": "^4.3.0",
35    "prismjs": "^1.25.0",
36    "react": "^17.0.1",
37    "react-dom": "^17.0.1",
38    "react-helmet": "^6.1.0",
39    "typeface-merriweather": "0.0.72",
40    "typeface-montserrat": "0.0.75"
41  },
42  "devDependencies": {
43    "prettier": "^2.4.1"
44  },
45  "homepage": "https://kolstadmagnus.no",
46  "keywords": [
47    "blog"
48  ],
49  "license": "0BSD",
50  "main": "n/a",
51  "repository": {
52    "type": "git",
53    "url": "git+https://github.com/gatsbyjs/gatsby-starter-blog.git"
54  },
55  "scripts": {
56    "build": "gatsby build",
57    "develop": "gatsby develop",
58    "format": "prettier --write \"**/*.{js,jsx,ts,tsx,json,md}\"",
59    "start": "gatsby develop",
60    "serve": "gatsby serve",
61    "clean": "gatsby clean",
62    "test": "echo \"Write tests! -> https://gatsby.dev/unit-testing\" && exit 1"
63  }
64}
657:11:59 PM: failed Building production JavaScript and CSS bundles - 20.650s
667:11:59 PM: error Generating JavaScript bundles failed
677:11:59 PM: Module build failed (from ./node_modules/url-loader/dist/cjs.js):
687:11:59 PM: Error: error:0308010C:digital envelope routines::unsupported
697:11:59 PM:     at new Hash (node:internal/crypto/hash:67:19)
707:11:59 PM:     at Object.createHash (node:crypto:130:10)
717:11:59 PM:     at getHashDigest (/opt/build/repo/node_modules/file-loader/node_modules/loader-utils/lib/getHashDigest.js:46:34)
727:11:59 PM:     at /opt/build/repo/node_modules/file-loader/node_modules/loader-utils/lib/interpolateName.js:113:11
737:11:59 PM:     at String.replace (<anonymous>)
747:11:59 PM:     at interpolateName (/opt/build/repo/node_modules/file-loader/node_modules/loader-utils/lib/interpolateName.js:110:8)
757:11:59 PM:     at Object.loader (/opt/build/repo/node_modules/file-loader/dist/index.js:29:48)
767:11:59 PM:     at Object.loader (/opt/build/repo/node_modules/url-loader/dist/index.js:127:19)
777:11:59 PM: ​
787:11:59 PM: ────────────────────────────────────────────────────────────────
797:11:59 PM:   "build.command" failed                                        
807:11:59 PM: ────────────────────────────────────────────────────────────────
817:11:59 PM: ​
827:11:59 PM:   Error message
837:11:59 PM:   Command failed with exit code 1: npm run build
847:11:59 PM: ​
857:11:59 PM:   Error location
867:11:59 PM:   In Build command from Netlify app:
877:11:59 PM:   npm run build
887:11:59 PM: ​
897:11:59 PM:   Resolved config
907:11:59 PM:   build:
917:11:59 PM:     command: npm run build
927:11:59 PM:     commandOrigin: ui
937:11:59 PM:     publish: /opt/build/repo/public
947:11:59 PM:     publishOrigin: ui
957:11:59 PM:   plugins:
967:11:59 PM:     - inputs: {}
977:11:59 PM:       origin: ui
987:11:59 PM:       package: '@netlify/plugin-gatsby'
997:11:59 PM:   redirects:
1007:12:00 PM:     - from: /api/*
101      status: 200
102      to: /.netlify/functions/gatsby
103    - force: true
104      from: https://magnuskolstad.com
105      status: 301
106      to: https://kolstadmagnus.no
107  redirectsOrigin: config
108Caching artifacts
109node -v > .nvmrc
110

This should create a .nvmrc file containing the Node version (node -v) in it. When Netlify finds this file during the build process, it uses it as a base Node version so it installs all the dependencies accordingly.

The file is also useful to tell other contributors which Node version are you using.

Source https://stackoverflow.com/questions/70362755

QUESTION

App Store Connect Upload Error "You haven't been given access to cloud-managed distribution certificates"

Asked 2021-Dec-22 at 10:31

Uploading an iOS app to App Store Connect with Xcode (Automatically manage signing) and received this error:

Screenshot of Xcode error dialog

The following errors occurred while locating and generating signing assets. ...

Communication with Apple failed. You haven't been given access to cloud-managed distribution certificates. Please contact your team's Account Holder or an Admin to give you access. If you need further assistance, contact Apple Developer Program Support at https://developer.apple.com/support

I have checked:

  • the cert is installed and valid
  • I have access to Certificates, Identifiers & Profiles

ANSWER

Answered 2021-Oct-18 at 01:45

the cert is installed and valid

That doesn't matter. New in Xcode 13, if you choose Automatic signing, Apple tries to do cloud-based signing; it doesn't even see the certificate that's on your computer.

But you do not have the cloud-based signing privilege, so it fails.

You have two choices:

  • Get the privilege. It is really worth it, because cloud-based signing is great! It allows you to distribute from an archive to App Store Connect without having any distribution identity or distribution certificate at all. This totally solves the problem that there's only one distribution certificate at a time.

  • Switch to manual signing. Now the distribution certificate on your computer will be used. You'll need explicit access to the distribution profile too, obviously; the whole export resigning will be manual. That might be simplest if you're in a hurry.

Source https://stackoverflow.com/questions/69609859

QUESTION

Python 3.10 pattern matching (PEP 634) - wildcard in string

Asked 2021-Dec-17 at 10:43

I got a large list of JSON objects that I want to parse depending on the start of one of the keys, and just wildcard the rest. A lot of the keys are similar, like "matchme-foo" and "matchme-bar". There is a builtin wildcard, but it is only used for whole values, kinda like an else.

I might be overlooking something but I can't find a solution anywhere in the proposal:

https://docs.python.org/3/whatsnew/3.10.html#pep-634-structural-pattern-matching

Also a bit more about it in PEP-636:

https://www.python.org/dev/peps/pep-0636/#going-to-the-cloud-mappings

My data looks like this:

1data = [{
2          "id"     : "matchme-foo",
3          "message": "hallo this is a message",
4      },{
5          "id"     : "matchme-bar",
6          "message": "goodbye",
7      },{
8          "id"     : "anotherid",
9          "message": "completely diffrent event"
10      }, ...]
11

I want to do something that can match the id without having to make a long list of |'s.

Something like this:

1data = [{
2          "id"     : "matchme-foo",
3          "message": "hallo this is a message",
4      },{
5          "id"     : "matchme-bar",
6          "message": "goodbye",
7      },{
8          "id"     : "anotherid",
9          "message": "completely diffrent event"
10      }, ...]
11for event in data:
12    match event:
13        case {'id':'matchme-*'}: # Match all 'matchme-' no matter what comes next
14            log.INFO(event['message'])
15        case {'id':'anotherid'}:
16            log.ERROR(event['message'])
17

It's a relatively new addition to Python so there aren't many guides on how to use it yet.

ANSWER

Answered 2021-Dec-17 at 10:43

You can use a guard:

1data = [{
2          "id"     : "matchme-foo",
3          "message": "hallo this is a message",
4      },{
5          "id"     : "matchme-bar",
6          "message": "goodbye",
7      },{
8          "id"     : "anotherid",
9          "message": "completely diffrent event"
10      }, ...]
11for event in data:
12    match event:
13        case {'id':'matchme-*'}: # Match all 'matchme-' no matter what comes next
14            log.INFO(event['message'])
15        case {'id':'anotherid'}:
16            log.ERROR(event['message'])
17for event in data:
18    match event:
19        case {'id': x} if x.startswith("matchme"): # guard
20            print(event["message"])
21        case {'id':'anotherid'}:
22            print(event["message"])
23

Quoting from the official documentation,

Guard

We can add an if clause to a pattern, known as a β€œguard”. If the guard is false, match goes on to try the next case block. Note that value capture happens before the guard is evaluated:

1data = [{
2          "id"     : "matchme-foo",
3          "message": "hallo this is a message",
4      },{
5          "id"     : "matchme-bar",
6          "message": "goodbye",
7      },{
8          "id"     : "anotherid",
9          "message": "completely diffrent event"
10      }, ...]
11for event in data:
12    match event:
13        case {'id':'matchme-*'}: # Match all 'matchme-' no matter what comes next
14            log.INFO(event['message'])
15        case {'id':'anotherid'}:
16            log.ERROR(event['message'])
17for event in data:
18    match event:
19        case {'id': x} if x.startswith("matchme"): # guard
20            print(event["message"])
21        case {'id':'anotherid'}:
22            print(event["message"])
23match point:
24     case Point(x, y) if x == y:
25         print(f"The point is located on the diagonal Y=X at {x}.")
26     case Point(x, y):
27         print(f"Point is not on the diagonal.")
28

Source https://stackoverflow.com/questions/70111401

Community Discussions contain sources that include Stack Exchange Network

Tutorials and Learning Resources in Cloud

Share this Page

share link

Get latest updates on Cloud