[Closed] [DONE] Issue:Perfomance:Import & analysis:Elephant & PostgreSQL load CPU up to 100%

Hi there,

we'll have a try if it's reproducable on our side as well.

Thx,
L

Hi there,

I just put the data in and the import runs very smoothly using the latest Elephant builds:

294 imported files
non not imported files
298.054 imported hands
108 not imported files
8 minutes and 7 seconds import duration

As you can see our new import is fast as lightning.

The legacy analysis on the other hand is still working - as expected.
You may already know that we're currently working on a complete new analysis engine, the prerequisite for the new tourney HUD. And we will enhance it's performance as much as we already did for the import engine.

But at the moment the legacy analysis engine is a known bottle neck and the more data the legacy analysis engine gets the slower it runs.
Analysing smaller packages with a maximum of ~50.000 hands will run much better for the legacy.

Cheers,
L

PS: The legacy analysis of the 300.000 hands is nearly working an hour now and still running

Strange. I test in on too PCs with dual-core CPU & with Quad CPU. The both PCs shows the similar results for legacy analysis engine. ?(

What about CPU usage? As I see you use Quad CPU . Elephant & PostgreSQL should load CPU up to 90% during import period.

Hi there,

the usage stays is between 55 and 65%.
But there is indeed an errorneous file reported by the analysis:

ERROR: Exception caught in Void PS.Database.nHibernate.Repositories.HandRepository.Update(NHibernate.ISession, System.Collections.Generic.IEnumerable`1[PS.Poker.IHand])
-------------------- PS.Common.Database.DatabaseException --------------------
- Message: identifier of an instance of PS.Poker.Hand.Hand was altered from 9816976768_0_1_3_0 to 9816976768_0_1_0_0

Can you arrange and send us the original hand history holding the game with the above mentioned ID '9816976768'.

Thx,
L

PS: Analysis ist still running...

Unfortunally I cann`t get original HH but I request this hand via e-mail. the received hand has the same structure as in the file 218SH2010.11.0713.txt. Do you need claimed hand?

UPD: Screenshot from "Quad PC"

Hi stanisluv,

the cpu usage is related to the analysis, as there are issues analysing the hands.
In order to invesigate this further we need the originals.

And Elephant v0.97 is released, I suggest to switch to the latest build.

Cheers,
L

Hi Larres,

Ok, I agree with you about PartyPoker but what about PokerStars? I play on PokerStars & have the same issue.

Today I have tried to import about 73k hands from 600 files & got following results:

Import duration: 8min
Analysis duration: 2h 20min. ;(

The CPU usage due to first 8 mins(import+analysis): 90-100%

What do you think about this?

Hi stanisluv,

don't get me wrong. The slow analysis is a big issue here and we're indeed about to rewrite the whole analysis engine as a prerequisite for the new tourney HUD. And as soon as the new analysis engine is ready to use this bottle neck will be gone.

Cheers,
L

Forgot to notify: somebody up Elephant perfomance in v 0.98 & now I I have following results:

Hands: 76k in 635 files
Import duration: 8min
Analysis duration: 1h 50min.

Nice work guys!! ♠_thumbsup: ♠_thumbsup:
♠_thumbsup:

Hi stanisluv,

We will very likely publis the 0.99 in around 2 weeks time. Can you repeat that speed test with the 0.99? We have tweaked something on the legacy import part and this shall be a little faster now.

Ciao
Michael

Hi Michael,

Yes, of course. Thx for info & timeline.

Stanislav.

Hi stanisluv,

the v0.99 is already available through our blog and I'd like you to check the improved version for new results

Cheers,
L

Already checked & reported.

So I assume this one is done?!
Anyway the next version comes with a fix for the memory constumption.

L

Yes, you can.

okey dokey