kandi background
Explore Kits

pulsar | Apache Pulsar - distributed pub-sub messaging system | Pub Sub library

 by   apache Java Version: v2.10.0 License: Apache-2.0

 by   apache Java Version: v2.10.0 License: Apache-2.0

Download this library from

kandi X-RAY | pulsar Summary

pulsar is a Java library typically used in Messaging, Pub Sub, Kafka applications. pulsar has no bugs, it has no vulnerabilities, it has build file available, it has a Permissive License and it has high support. You can download it from GitHub, Maven.
Pulsar is a distributed pub-sub messaging platform with a very flexible messaging model and an intuitive client API. Learn more about Pulsar at https://pulsar.apache.org.
Support
Support
Quality
Quality
Security
Security
License
License
Reuse
Reuse

kandi-support Support

  • pulsar has a highly active ecosystem.
  • It has 10675 star(s) with 2795 fork(s). There are 422 watchers for this library.
  • There were 5 major release(s) in the last 12 months.
  • There are 1695 open issues and 3122 have been closed. On average issues are closed in 83 days. There are 361 open pull requests and 0 closed requests.
  • It has a positive sentiment in the developer community.
  • The latest version of pulsar is v2.10.0
pulsar Support
Best in #Pub Sub
Average in #Pub Sub
pulsar Support
Best in #Pub Sub
Average in #Pub Sub

quality kandi Quality

  • pulsar has 0 bugs and 0 code smells.
pulsar Quality
Best in #Pub Sub
Average in #Pub Sub
pulsar Quality
Best in #Pub Sub
Average in #Pub Sub

securitySecurity

  • pulsar has no vulnerabilities reported, and its dependent libraries have no vulnerabilities reported.
  • pulsar code analysis shows 0 unresolved vulnerabilities.
  • There are 0 security hotspots that need review.
pulsar Security
Best in #Pub Sub
Average in #Pub Sub
pulsar Security
Best in #Pub Sub
Average in #Pub Sub

license License

  • pulsar is licensed under the Apache-2.0 License. This license is Permissive.
  • Permissive licenses have the least restrictions, and you can use them in most projects.
pulsar License
Best in #Pub Sub
Average in #Pub Sub
pulsar License
Best in #Pub Sub
Average in #Pub Sub

buildReuse

  • pulsar releases are available to install and integrate.
  • Deployable package is available in Maven.
  • Build file is available. You can build the component from source.
  • Installation instructions, examples and code snippets are available.
  • It has 444636 lines of code, 28552 functions and 3482 files.
  • It has medium code complexity. Code complexity directly impacts maintainability of the code.
pulsar Reuse
Best in #Pub Sub
Average in #Pub Sub
pulsar Reuse
Best in #Pub Sub
Average in #Pub Sub
Top functions reviewed by kandi - BETA

kandi has reviewed pulsar and discovered the below as its top functions. This is intended to give you an instant insight into pulsar implemented functionality, and help decide if they suit your requirements.

  • Creates a BitSetRecyclable from the specified ByteBuffer .
    • Prints statistics for a topic .
      • Handles a command producer .
        • Update a function .
          • Update a source .
            • Updates an existing sink .
              • Calculate cursor back cursors counters .
                • Returns command to execute .
                  • Truncate ledgers .
                    • Returns the internal statistics for this topic .

                      Get all kandi verified functions for this library.

                      Get all kandi verified functions for this library.

                      pulsar Key Features

                      Horizontally scalable (Millions of independent topics and millions of messages published per second)

                      Strong ordering and consistency guarantees

                      Low latency durable storage

                      Topic and queue semantics

                      Load balancer

                      Designed for being deployed as a hosted service: Multi-tenant Authentication Authorization Quotas Support mixing very different workloads Optional hardware isolation

                      Keeps track of consumer cursor position

                      REST API for provisioning, admin and stats

                      Geo replication

                      Transparent handling of partitioned topics

                      Transparent batching of messages

                      Build Pulsar

                      copy iconCopydownload iconDownload
                      $ mvn install -DskipTests
                      

                      Minimal build (This skips most of external connectors and tiered storage handlers)

                      copy iconCopydownload iconDownload
                      mvn install -Pcore-modules,-main -DskipTests
                      

                      Build custom docker images

                      copy iconCopydownload iconDownload
                      mvn clean install -DskipTests
                      mvn package -Pdocker,-main -am -pl docker/pulsar-all -DskipTests
                      

                      error 404 when try to make oauth in pulsar stream native cluster

                      copy iconCopydownload iconDownload
                      pulsar+ssl://sn-myinstance.mycluster.snio.cloud:6651 
                      
                      client = pulsar.Client(args.service_url, authentication=AuthenticationOauth2(args.auth_params))
                      
                      pulsar+ssl://sn-myinstance.mycluster.snio.cloud:6651 
                      
                      client = pulsar.Client(args.service_url, authentication=AuthenticationOauth2(args.auth_params))
                      

                      Apache Pulsar: Access state storage in LocalRunner not working

                      copy iconCopydownload iconDownload
                      2022-02-07T11:09:04,916-0800 [main] WARN  com.scurrilous.circe.checksum.Crc32cIntChecksum - Failed to load Circe JNI library. Falling back to Java based CRC32c provider
                      >>> PRODUCER SENT
                      2022-02-07T11:09:05,267-0800 [public/default/TestFunction-0] INFO  org.apache.pulsar.functions.instance.state.BKStateStoreProviderImpl - Opening state table for function public/default/TestFunction
                      2022-02-07T11:09:05,279-0800 [client-scheduler-OrderedScheduler-7-0] INFO  org.apache.bookkeeper.clients.SimpleStorageClientImpl - Retrieved table properties for table public_default/TestFunction : stream_id: 1024
                      
                      2022-02-07T11:09:05,527-0800 [pulsar-client-io-1-2] INFO  org.apache.pulsar.client.impl.ConsumerImpl - [test-topic-input][Test Function Sub] Subscribed to topic on localhost/127.0.0.1:6650 -- consumer: 0
                      >>> GOT input Hello World!
                      

                      Pass private key as header in curl PUT returning error for illegal character

                      copy iconCopydownload iconDownload
                      Authorization: Bearer $AUTHORIZATION
                      
                      bin/pulsar tokens create --private-key file:///path/to/my-private.key \
                                  --subject test-user
                      

                      How to pass authorization key in shell script curl command without header

                      copy iconCopydownload iconDownload
                      AUTHORIZATION=$(cat $REPO_ROOT/pulsar/tls/broker/broker.key.pem)
                      

                      Event-time Temporal Join in Apache Flink only works with small datasets

                      copy iconCopydownload iconDownload
                      WATERMARK FOR MUT_TS AS MUT_TS - INTERVAL '2' MINUTE
                      

                      Why does the data streamed using the Java client appear as encoded on Apache Pulsar?

                      copy iconCopydownload iconDownload
                      Schema.AVRO(DavisMessage.class)
                      

                      Pulsar client thread balance

                      copy iconCopydownload iconDownload
                      PulsarClient client = PulsarClient.builder()
                          .serviceUrl("pulsar://localhost:6650")
                          .ioThreads(16)
                          .connectionsPerBroker(16)
                          .create();
                      
                      

                      Pulsar function fails to deserialize message because of wrong schema type (JSON instead of AVRO)

                      copy iconCopydownload iconDownload
                      Map<String, ConsumerConfig> inputSpecs = new HashMap<String, ConsumerConfig> ();
                      inputSpecs.put("persistent://orders/inbound/food-orders", 
                          ConsumerConfig.builder().schemaType("avro").build());
                      FunctionConfig functionConfig = 
                          FunctionConfig.builder()
                              ...
                              .inputSpecs(inputSpecs)
                              ...
                              .build();
                      
                      
                      inputSpecs:
                       $topicName:
                        schemaType: AVRO
                      
                      Map<String, ConsumerConfig> inputSpecs = new HashMap<String, ConsumerConfig> ();
                      inputSpecs.put("persistent://orders/inbound/food-orders", 
                          ConsumerConfig.builder().schemaType("avro").build());
                      FunctionConfig functionConfig = 
                          FunctionConfig.builder()
                              ...
                              .inputSpecs(inputSpecs)
                              ...
                              .build();
                      
                      
                      inputSpecs:
                       $topicName:
                        schemaType: AVRO
                      

                      Pulsar: If a message gets nack'd (negativeAcknowledge()) when will it be redelivered?

                      copy iconCopydownload iconDownload
                      Consumer<byte[]> consumer = client.newConsumer()
                          .topic("my-topic")
                          .subscriptionName("my-sub")
                          .negativeAckRedelivery(10, TimeUnit.SECONDS)
                          .subscribe()
                      

                      How to organize multiple Dockerfiles, docker-compose.yaml and .dockerignore

                      copy iconCopydownload iconDownload
                      COPY requirements.txt ./
                      
                      version: '3.8'
                      services:
                         producer:
                           build: ./python_producer
                           environment:
                             - RABBITMQ_HOST=rabbitmq
                         consumer:
                           build: ./python_consumer
                           environment:
                             - RABBITMQ_HOST=rabbitmq
                         rabbitmq:
                           image: rabbitmq:3
                           hostname: rabbitmq # RabbitMQ is very unusual in needing to set this
                      
                      COPY python_producer/requirements.txt ./
                      
                      version: '3.8'
                      services:
                         producer:
                           build:
                             context: .
                             dockerfile: python_producer/Dockerfile
                           environment:
                             - RABBITMQ_HOST=rabbitmq
                         consumer:
                           build:
                             context: .
                             dockerfile: python_consumer/Dockerfile
                           environment:
                             - RABBITMQ_HOST=rabbitmq
                         rabbitmq:
                           image: rabbitmq:3
                           hostname: rabbitmq # RabbitMQ is very unusual in needing to set this
                      
                      COPY requirements.txt ./
                      
                      version: '3.8'
                      services:
                         producer:
                           build: ./python_producer
                           environment:
                             - RABBITMQ_HOST=rabbitmq
                         consumer:
                           build: ./python_consumer
                           environment:
                             - RABBITMQ_HOST=rabbitmq
                         rabbitmq:
                           image: rabbitmq:3
                           hostname: rabbitmq # RabbitMQ is very unusual in needing to set this
                      
                      COPY python_producer/requirements.txt ./
                      
                      version: '3.8'
                      services:
                         producer:
                           build:
                             context: .
                             dockerfile: python_producer/Dockerfile
                           environment:
                             - RABBITMQ_HOST=rabbitmq
                         consumer:
                           build:
                             context: .
                             dockerfile: python_consumer/Dockerfile
                           environment:
                             - RABBITMQ_HOST=rabbitmq
                         rabbitmq:
                           image: rabbitmq:3
                           hostname: rabbitmq # RabbitMQ is very unusual in needing to set this
                      
                      COPY requirements.txt ./
                      
                      version: '3.8'
                      services:
                         producer:
                           build: ./python_producer
                           environment:
                             - RABBITMQ_HOST=rabbitmq
                         consumer:
                           build: ./python_consumer
                           environment:
                             - RABBITMQ_HOST=rabbitmq
                         rabbitmq:
                           image: rabbitmq:3
                           hostname: rabbitmq # RabbitMQ is very unusual in needing to set this
                      
                      COPY python_producer/requirements.txt ./
                      
                      version: '3.8'
                      services:
                         producer:
                           build:
                             context: .
                             dockerfile: python_producer/Dockerfile
                           environment:
                             - RABBITMQ_HOST=rabbitmq
                         consumer:
                           build:
                             context: .
                             dockerfile: python_consumer/Dockerfile
                           environment:
                             - RABBITMQ_HOST=rabbitmq
                         rabbitmq:
                           image: rabbitmq:3
                           hostname: rabbitmq # RabbitMQ is very unusual in needing to set this
                      
                      COPY requirements.txt ./
                      
                      version: '3.8'
                      services:
                         producer:
                           build: ./python_producer
                           environment:
                             - RABBITMQ_HOST=rabbitmq
                         consumer:
                           build: ./python_consumer
                           environment:
                             - RABBITMQ_HOST=rabbitmq
                         rabbitmq:
                           image: rabbitmq:3
                           hostname: rabbitmq # RabbitMQ is very unusual in needing to set this
                      
                      COPY python_producer/requirements.txt ./
                      
                      version: '3.8'
                      services:
                         producer:
                           build:
                             context: .
                             dockerfile: python_producer/Dockerfile
                           environment:
                             - RABBITMQ_HOST=rabbitmq
                         consumer:
                           build:
                             context: .
                             dockerfile: python_consumer/Dockerfile
                           environment:
                             - RABBITMQ_HOST=rabbitmq
                         rabbitmq:
                           image: rabbitmq:3
                           hostname: rabbitmq # RabbitMQ is very unusual in needing to set this
                      

                      Community Discussions

                      Trending Discussions on pulsar
                      • error 404 when try to make oauth in pulsar stream native cluster
                      • Kubernetes Statefulsets: Restart all pods concurrently (instead of in sequence)
                      • How to work with protobuf schema messages in Python Apache pulsar pulsar-client package?
                      • Apache Pulsar: Access state storage in LocalRunner not working
                      • non-persistent message is lost when throughput is high
                      • ProducerBlockedQuotaExceededError: Cannot create producer on topic with backlog quota exceeded
                      • Pass private key as header in curl PUT returning error for illegal character
                      • How to pass authorization key in shell script curl command without header
                      • Event-time Temporal Join in Apache Flink only works with small datasets
                      • does pulsar support multiple bookkeeper replicas in different clusters
                      Trending Discussions on pulsar

                      QUESTION

                      error 404 when try to make oauth in pulsar stream native cluster

                      Asked 2022-Mar-24 at 21:35

                      Hello I am triying to connect to apache pulsar cluster using stream native, I don't have problems with token oauth, but when I try to make Oauth I always get malformed responde or 404 I am using curl and python client, and following their instructions,like this.

                      params = '''
                      {
                          "issuer_url": "https://auth.streamnative.cloud/",
                          "private_key": test.json",
                          "audience": "urn:sn:pulsar:test:test"
                      }
                      '''
                      
                      pulsar_client = pulsar.Client(
                          PULSAR_URL,
                          authentication=AuthenticationOauth2(params)
                      )
                      

                      and the error is the following:

                      AuthOauth2:223 | Response failed for getting the well-known configuration https://auth.streamnative.cloud/. response Code 404
                      

                      But the params and url I get from https://console.streamnative.cloud/test/test/clients, but nothing.

                      Any Idea about how can I connect to the cluster with the oauth?

                      EDIT

                      With client the credentials works, looks like a bug in python package of pulsar 2.9.1.

                      Thanks

                      ANSWER

                      Answered 2022-Mar-24 at 20:25

                      You may need the full path to the private key. make sure it has permissions.

                      also make sure your audience is correct

                      what is pulsar URL format?

                      pulsar+ssl://sn-myinstance.mycluster.snio.cloud:6651 
                      
                      client = pulsar.Client(args.service_url, authentication=AuthenticationOauth2(args.auth_params))
                      

                      https://docs.streamnative.io/cloud/stable/connect/client/connect-python

                      https://github.com/streamnative/examples/blob/master/cloud/README.md

                      Also make sure you gave the userid associated with your key permissions.

                      If that doesn't work, chat with me on the Pulsar slack.

                      Source https://stackoverflow.com/questions/71577989

                      Community Discussions, Code Snippets contain sources that include Stack Exchange Network

                      Vulnerabilities

                      No vulnerabilities reported

                      Install pulsar

                      Compile and install individual module.
                      Java JDK 11 or JDK 8
                      Maven 3.6.1+
                      zip
                      Run Individual Unit Test:.
                      Docker images must be built with Java 8 for branch-2.7 or previous branches because of issue 8445. Java 11 is the recommended JDK version in master/branch-2.8. This builds the docker images apachepulsar/pulsar-all:latest and apachepulsar/pulsar:latest. After the images are built, they can be tagged and pushed to your custom repository. Here's an example of a bash script that tags the docker images with the current version and git revision and pushes them to localhost:32000/apachepulsar.
                      Apache Pulsar is using lombok so you have to ensure your IDE setup with required plugins.
                      Refer to the docs README.

                      Support

                      Pulsar Translation

                      DOWNLOAD this Library from

                      Find, review, and download reusable Libraries, Code Snippets, Cloud APIs from
                      over 430 million Knowledge Items
                      Find more libraries
                      Reuse Solution Kits and Libraries Curated by Popular Use Cases
                      Explore Kits

                      Save this library and start creating your kit

                      Explore Related Topics

                      Share this Page

                      share link
                      Reuse Pre-built Kits with pulsar
                      Consider Popular Pub Sub Libraries
                      Try Top Libraries by apache
                      Compare Pub Sub Libraries with Highest Support
                      Compare Pub Sub Libraries with Highest Quality
                      Compare Pub Sub Libraries with Highest Security
                      Compare Pub Sub Libraries with Permissive License
                      Compare Pub Sub Libraries with Highest Reuse
                      Find, review, and download reusable Libraries, Code Snippets, Cloud APIs from
                      over 430 million Knowledge Items
                      Find more libraries
                      Reuse Solution Kits and Libraries Curated by Popular Use Cases
                      Explore Kits

                      Save this library and start creating your kit

                      • © 2022 Open Weaver Inc.