large number of Channels Kafka -

-

I was wondering if there is a limit to kafa or starts at a slow pace (due to GC or other reasons) We have a large number of channels. We have a large amount of data that we will send through Kafka (more than 2b data points). We were thinking of launching around 1600 channels.

Do we have such a large channel in Kaffa when it comes to any issue? Similarly, are you getting problems with the local DC replication with a large number of channels and in the end, if we are using mirrmakers for DC replication with such a large number of channels in the near future

There are no signals on the number of topics only in the KPKK.

However, as long as possible Store information about the subject in the Oyster (// broker / subjects /), and Zukipar has a maximum limit of 1 MB on the node size, there can be only a limited number of topics, besides Kafka Broker / var / kafka Store data for different topics in / data / if there are too many sub-players in / var / kafka / data / then the performance may be affected

I have not tried thousands of themes, but Kafa ka Six works fine for my purposes with hundred subjects were the only area where I have problems, high-end consumer was the dynamic theme built using. It requires a client re-connection to select new topics on all the consumer boxes, it spent time re-balancing the consumer (which sometimes failed, preventing reading from certain topics) as a result of switching over to an ordinary consumer Had to be mindful about the coordination of reading in my code.

I recommend creating a simple test app that generates some random data for the number of topics you go forward and verify that the display is acceptable.

Comments

Post a Comment

Popular posts from this blog

java - ImportError: No module named py4j.java_gateway -

-
I'm trying to call the Python using the Java program py4j . I've been installing the plug-in Eclipse and test name Piidvi project. I'm trying to execute the following part of the code found on py4j webpage: Import from py4j.java_gateway to JavaGateway, java_import gateway = JavaGateway () jvm = gateway.jvm java_import (jvm, '' Org.eclipse Kkorkrisorsej. * ") Vrkspes_rut = Jvankresourkesplginkgetvrkspas (). GetRoot (Gateway .help (workspace_root, '* Projects *') project_names = [project.getName () (for projects workpace_root.getProjects))] print (Projekt_nam) But I There is an error in import. I have checked that the P4JJ is present in the Jar Eclipse plugin directory. Can anyone help please? I had to install the py4j application
Read more

python - Receiving "KeyError" after decoding json result from url -

-
I am new to Python I am trying to parse JSON result from a URL. Basically, I was using the following: response = urllib.request.urlopen (url) json_obj = json.load (response) It should be a stroke "str 'not' bytes' in the lines of a given" JSON object, so after searching on the StackoverView Flo, I decode the response in this way: F = urllib.request.urlopen (Url) charset = f.info (). Get_param ('charset', 'utf8') data = f.read () decoded = json.loads (data.decode (charset)) If I print "decode" I is as follows: { 'link': { 'summary data': 'https: // localhost / piwebapi / streams / p0_7qHaW4UHU-RlCaz8tpasAAQAAAAU0hJTExNQU42NDIwXFNJTlVTT0lE / summary' 'value': 'https: // localhost / Piwebapi / streams / P0_7qHaW4UHU-RlCaz8tpasAAQAAAAU0hJTExNQU42NDIwXFNJTlVTT0lE / price ',' InterpolatedData ':' https: // localhost / Piwebapi / streams / P0_7qHaW4UHU-RlCaz8tpasAAQAAAAU0hJTE...
Read more

.net - Creating a new Queue Manager and Queue in Websphere MQ (using C#) -

-
"itemprop =" text "> I am writing applications that use WebSphere MQ to send messages. My unittests (for flow), I want to verify that I have put the correct message on the response queue. I am trying to figure out how to do this. My main obstacle is that I think it might be scary to clear the queue before running in my queue because the same queue can be used by other applications, I thought a decent solution would be a new line The manager will remove the queue for my solidarity and after using it. So my question is: Is it possible to use C # by using queue manager and queue? After the For future reference and future people, who want to make queues. I thought how to make and remove IBM MQ QE (not quenhers) with PCM messaging. It is not very simple, but it can be done. We have implemented it in a library and it is being used to create and delete commands before and after integration tests. The most important part of the code in this library is shown in the...
Read more