SIte load process / performance

[#1]

I beleive the maintenance window does occur in the early hours every wednesday.
The 'whos online' panel has been identified as a pain point due to the amount of requests it needs to make to get information from each user. I either need to redevelop it to use a single api call to get all the info or disable it. I will need to speak to @jono about disabling it temporarily but the panel does add a bit of life to the pages that it exists on.

[#2]

Please see screen recording to follow...

In the screen recording (attached if possible or sent via some other means TBA) I do the following...

1. Load the forum home page
2. Select PN feed back (mistake)
3. Select Forum feed back
4. Select this topic
5. Seek to post to this topic

Please note the following frames in the video...

1. 00:00:15.25 - Page scrolls to bottom (who is on line) and a grid can be seen at the bottom of the page
2. 00:00:15.75 - Page scrolls to top
3. 00:00:30.50 - board content index starts to load - note missing Plusnet logo … implies it is downloaded on every page load rather than being brought from cache
4. 00:00:31.75 - logo starts to load in parts - completes 32.25
5. 00:00:33.25 - search box magnifying glass appears
6. 00:00:36.00 - FORUM button size twitches - restored at 36.75
7. 00:00:35.50 - Who is on line scrolls into view, very clearly showing a "Submit query block" being built at the very bottom
8. Similar sequence for the above happens on the forum board index page loads and topic load … though note slightly different load sequences for logo and page title
9. 00:02:17.45 - Reply screen loads
10. 00:02:19.25 - screen scrolls up
11. 00:02:19.50 - Post box border vanishes
12. 00:02:20.75 - Macro button get focus (just made me wonder if the issues I see are related to having macros enabled?)
13. 00:02:22.00 - Post inox size changes (causes stuff to re-render)
14. 00:02:22.25 - Oh it changes size back again, regains the border, displays some edit icons and re-renders
15. 00:02:24.00 - The rest of the edit icons appear - are these being re-served rather than coming from cache?
16. 00:02:26.50 - The screen scrolls to the bottom again - note submit query dross at the bottom of the page
17. 00:02:26.75 - Scroll back to top … in the next frame MACROS loses focus
18. 00:02:32.75 - I can finally type into the box 15.5 seconds after the screen completed its load

I suggest that the above illustrates that the client side page processing is not swift, can be cleaned up and that cache utilisation can be markedly optimised.

The attached file is a MP4 file renamed .jpeg so as I can by pass the upload filters!  

That did not work, the filters are not that easily fooled!  Need a different solution!

[#3]

Need to spend money to convert to a format acceptable to the forum … so sent to @jaread83 via another route!!

[#4]

I have just spent a bit of time refactoring the online users module. The problem with it was this -
1 api call to get the total number of online users
1 api call to get the list of online users (max 100 items)
3 api calls per user to get rank id, rank name and avatar.

So if there are 100 items in the list, this would make 302 requests each time the module loads.

I have refactored the component so that it comes down to the following:
1 api call to get the total number of online users (this has been left in)
1 api call to get all online users - the returned data includes the rank name, rank id, username, link to profile page and the avatar.

So we will be going from 302 requests to 2.

The reason it wasn't like this before is because the API that gets all online users did not support fetching each avatar, rank id and rank name - all i could get was a list of users with basic info. Then in the loop through and make a request each time - a bit of a pain to be honest. But a recent update made it so i can get all the required data from the API easily and in 1 request. I am just testing it out now.

[#5]

Hit a snag and the new code is showing duplicate users (probably counting multiple sessions). Will get this resolved asap - hopefully this will speed up a lot of the main pages.

[#6]

@jaread83,

Did the screen recording help any?

Are these calls all client - server or are they in the server side page build? If client side then this feels like a clunky approach putting a sizeable payload client side and driving up band width utilisation...

Are the avatars cached?

[#7]

The requests are all client side from the component I have built. Lithium provide two API methods, one is URL based (V1) and the other is a MySQL based SELECT statement (V2). The problem with V1 is that it is rigid and provides almost everything but is cut down a bit - for example, you can get all 'online and registered' users via a URL but it does not include avatar images or ranks. So in my component, to get this information, I have to make another API call using the UserID to look up that information.
Now - API V2 lets me select whatever I want from the database tables in a nice little SQL style SELECT method, I can select only the information I need and bring back everything in one single request. Sounds good right? Well that would be my optimism when I did some research into it and recoded the component yesterday to use V2. It worked great on the staging site but when testing it with actual 'online' users, it seemed to be returning duplicates for absolutely no reason - the only thing I can think of is that it includes all online sessions. So users like Gandalf was appearing 7 times - probably because he has multiple tabs open and is also using the LSR CRM to interact with the forums. Not ideal. I am currently trying to resolve it via Lithium support and also speaking with other developers on other Lithium communities to see if I can get this resolved and use V2 api to condense the requests to only a few.
It is my strong beleif that the slowness of the community is linked to the online users component. With the amount of requests it is making compared to others, it makes sense to fix it like I have described above and failing that, another solution will be needed. One that doesn't have the mod, staff, admin and SU labels and also no avatars - something I feel gives a bit of life to the homepage.

[#8]

@jaread83 This might be a red herring, but the only slow loading I get is from clicking on a board heading - always takes 5-7 seconds of 'waiting for 'community.plus.net' before anythings happens, then everything loads too quickly for me to read the message at the bottom left of the screen (using FF63.0.1 64bit)

[#9]

something I feel gives a bit of life to the homepage.

I agree with that sentiment … maybe the answer is to display this ONLY on the home page, not every board index (topic list) page?

[#10]

@jaread83

My use of the forum is based around the "Unread Topics" page so I rarely see the online user list.

I go from topic to topic via that page. I click a topic, interact in some way or not, then either go back  in the browser or click the "Unread topic" button on the thread page.

And I get the same slowdowns that affect others.

Edit: I've had a thought (dangerous I know) and that what Jaread was saying is that the amount of people calling the online user list is the "real issue" TM. So that load is increasing the server  load hence slowing down the rest of us. Now I understand. Can take a while some times

[#11]

I have gone ahead and removed the component from everywhere except the homepage. Hopefully this will alleviate some of the requests being made to the server by a lot.
I will update the component itself when i get an answer from Lithium support about it.

[#12]

I think there are multiple opportunities to improve performance. The right cache lifetime settings for static objects (any graphic) would make a significant difference to page loading and bandwidth utilisation.

Page rendering appears clunky.

I think (out of shared frustration?) that Jack only comments on the bits he can tweak - the skin / client side automation given that Lithium seem reluctant or unable to optimise the flesh and bones of the technology stack.

Real shame I could not upload the diagnostic video ... it’s painful seeing all of the messing around which happens ... which when connected on a gigabit LAN in a test lab on a high spec PC is not discernible.

Very rarely does extensive real world user environment testing take place outside of professional commercial software development shops. I recall not getting anything passed for release which did not function proficiently in the EU environment ... no matter how good it looked in the QA labs.

[#13]

Not good news.    Back to 4 seconds to 'like' a post.

[#14]

@jaread83 Morning Jack. Sorry to disturb you -again- but clicking on a board header is resulting in a 'waiting for community.plus.net' delay of anything from 8 to 15 seconds again, 4-6 seconds to open a thread. Any idea why this happens?

I have done my usual check using the TT and Virgin communities, and they as usual are almost instant.

[#15]

@jab1  +1

It started sometime yesterday evening?