I have a requirement where there are around 800 topic consumers and one publisher, Can you help send your configuration, i am more interested in the memory setting you had done to handle 5000 clients.

For my case messages are getting struck for some subscribers.
At least post some of the important tweaks you had done for ur scenario, i will change accordingly for topics.

Thanks & Regards,
Mahesh

The queue dispatch thread used to be per destination, but in recent 5.3.x versions its using a shared thread pool factory and respects the dedicated task runner configuration.

In terms of the selector/vs partitioned destination question, the latter (partitioning) will be better as the selectors are limited by the pagedInMessages window.

On the support question, thats very exciting. I would say use this form when your ready to go down that road

Cheer’s
/Dave

We are using queues…. not topics…

Yes I think I will start posting a whole batch of questions to the mailing list forum very soon…

Now, interestingly when I reverted the settings (OptimizedDispatch etc)
back to the “vanilla” Out of the Box configuration, the crash error went away…

and here is what is pleasantly surprising.. I am actually able to have and sustain 5000 clients (each a producer and a consumer) with just plain TCP (not NIO) and those plain original settings (just with RAM values increased all over the place)

Are there any disadvantages of having it this way?
Or we actually should to try get working the recommendations that this post recommends…

Also, I should mention that we so far tried to have one IN queue and one OUT queue and have all of 5000 clients use them… and do a selector on “WHERE CustomeHeaderCLIENTID=’BLAHBLAH’”… but it seem that the consumers then becomes a serious bottleneck when the queue has a backlog (10-20K messages queued up)

Am I right in my understanding that this is what should be expected because the queue is not indexed in any way by this customer header we use for selector and therefor each consumer session does a FULL SCAN of sorts and potentially locks and blocks and when we have 5000 of this it becomes a bottleneck….

so we are going to change it now so that we have a dedicated queue for each consumer…

(and as far as I understand ActiveMQ now _can_ have 5000 queues easily)?

Are we on a right path here?

i.e. would you recommend one huge queue and 5000 clienst doing a “selector” on it or 5000 dedicated queues instead…

are there some “golden rules” regarding kind of considerations?

Thank you!

cheers,
O.K.
BTW, we are seriously contemplating getting a help from FUSE as we are a big PROGRESS customer already (for SonicMQ)… at least as an a consulting/training engagement (btw, by any chance maybe you or other ActiveMQ committers from FUSE feel like traveling to Australia? – that would be great if we got get one of the actual engineers to interrogate here

In terms of the thread counts are you using queue’s or topics? I just realized I just set a destination policy on the queue, but if your using topics you may need to also setup a destination policy on the topic. Remember to make sure you enabled optimizedDispatch.

On the crash, would need to see the stack from the hs_err_pid, but you could also try mailing the activemq user list for that one. The folks on the list are usually very responsive.

/Dave

It is all very clear now.

One last question:

re:
>4) Use a broker with a version > 5.3.0.3

I downloaded just a month ago and looked again now and also in the console
when it start and I don’t see “5.3.0.3″ anywhere… just “”5.3.0″…

where should I look to confirm that it is “….0.3″ (assuming you actually meant that)?

Also, I guess somewhat important – what Java version is recommended (we still use 1.5)

The reason I want to be very pedantic is that I am following the instructions from here and configured everything as suggested, but I still see about 2 threads per connection as opposed to 1 thread per 16 connections…

so does not seem that we got it quite right yet…

and also I am quite consistently getting a rather scary java crash
- have you ever seen anything like that?

#
# An unexpected error has been detected by HotSpot Virtual Machine:
#
# SIGBUS (0xa) at pc=0xffffffff7ee00c10, pid=20145, tid=3611
#
# Java VM: Java HotSpot(TM) 64-Bit Server VM (1.5.0_16-b02 mixed mode)
# Problematic frame:
# C [libc_psr.so.1+0xc10]
#
# An error report file with more information is saved as hs_err_pid20145.log
#
# If you would like to submit a bug report, please visit:
# http://java.sun.com/webapps/bugreport/crash.jsp

Please help!

Thank you!
cheers,
O.K.

No, you can just enable the nio transport and it will handle all incoming connections using nio.

>And also – what connection url should the clients use – “nio://localhost:62828?useQueueForAccept=false”

Just the regular tcp:// transport url for the producers and consumers. The nio is configuring how the incoming connection is handled on the broker side.

> And also, we are connecting from thousands of .NET clients using “NMS”… and there is no protocol option of “nio:” in NMS…
> is it ok to use just “tcp:” then?

Yes absolutely.

>as far as I understand NIO is still TCP and from the client’s side it should be transparent whether its handled using nio or “old io” on the server side?

Yep thats right. Its not a client side concern. This is all on the server side. We are trying to minimize the number of broker threads created when you have a large incoming connection load.

Hope that helps
/Dave