iPad Dashboards Dying - Troubleshooting?

Cory_Booth · June 11, 2024, 5:24am

I have been battling errors recently where my dashboards (I have three) - previously running perfectly on iPad gen 2 (x2) and iPad gen 5 (x1) started hanging overnight. Sometimes they would hang within hours of resetting.

I semi correlated this with an update to my Kiosk App - Kiosker Pro. They did a massive UI change.

But over time, I tried several other Kiosk apps and even Guided Access. But nothing seemed to help.

I was going down tbe road of replacing Safari as the default browser hoping the Kiosk apps would use this other browser but decided maube it was iOS and the iPads.

So I bought a lot of three iPad Gen 6…. And oddly they are goving me the same issues.

What I have noticed is a toast message at rhe bottom of my ipads (usually) saying something about “Socket”

I see Socket Connected when things are fine and Socket Status Unknown when the disboard gets stuck loading and Socket Disconnected when it seem the Dashboard has gone stale

I have read on here this seems to be a known issue?

Is there any advice for an iPad user? Settings to adjust? Logs to review?

josh · June 11, 2024, 4:21pm

Do you recall the approximate date? Looks like you posted about the Kiosker update on March 7th.

I can’t think of any changes we made after the introduction of the realtime socket connection in early 2024 that could contribute to this. But I estimate that most devices running the production site cutover to the realtime socket connection in mid-late February.

Can you clarify what’s a known issue? Perhaps a link to a thread?

The Socket Connected messages is normal. If the device gets stuck with a Socket Disconnected message, that’s not good as it means your dashboard is not getting realtime updates.

Since you have a variety of devices available for testing, have you tried creating a separate ‘test’ SharpTools account?

It’s not clear exactly what you meant by “started hanging overnight”, but that might determine what type of content you would need for testing. If it’s where you are clearly seeing an error like “Socket Disconnected”, then even a blank dashboard or just some dashboard with static content could work.

If it’s actions stopping working and realtime data stopping working, then perhaps just a few variables to test with (I’m happy to temporarily tag the test account with free premium access to testing this).

What I’m wondering is if perhaps there’s some content either on the dashboard or in the account itself which is causing things to get funky. At the very least, if we can rule that out, it helps us focus our efforts elsewhere.

Cory_Booth · June 13, 2024, 1:13am

So in this case, this has actually been going on for a while - even before this. However, because this was the only significant change in my configuration, I thought it could have been due to this update. I was a beta tester for Kiosker Pro - so I had advanced access to this update as well.

There are a few threads, but this one captures most of what I was understanding:
Background network disconnected - Reload - SharpTools.io (web) - SharpTools Community

I have not created a Test Account - honestly I’ve just been “dealing with it” hoping for an iOS upgrade or something…

In regards to how it manefests itself:

Here, the dashboard is no longer “live”

Here, this is what I typically wake up to after overnight - but this happens randomly even during the day. Sometimes I can go 1-2 days, but eventually all three iPads will wind up looking like this - I didn’t get a good photo but the toast says “Socket Status Unknown” and there is a “loading” animation that just sits there.

As an alternative - I tried downloading Brave and making it the “default” browser - no help.
I bought “Kiosk Pro Plus” - no help
I bought “tKiosk” - no help
I tried “Kiosk - Fullscreen Broswer” - no help
I tried “URL Bookmark” / guided access - no help

What I haven’t tried yet is using Brave as a “bookmark” and guided access…

i did get this replay from Kiosker Pro:
“We are aware of memory leaks on some legacy devices. Our investigation points at memory leaks in the Apple WebKit engine. We have filed a bug report to Apple and they have responded that tey are aware of the issue. No solution is available at the moment. The issue seems quite hard to pinpoint since crashes only happens on some devices. We have a iPad Air 2 running a Home Assistant Dashboard for months without crashing.”

I owe them some logs - which I will be working on collecting…

josh · June 13, 2024, 2:56am

Do you have a rough idea of when it might have started?

Thanks for the feedback. That particular thread is a bit unique as it was a really early introduction of the background worker where it wasn’t actually doing anything (intentionally). And this was actually showing an error about the worker itself which theoretically shouldn’t ever happen and was really intended as a failsafe.

We refined the notification so it only shows when the background worker is really dead.

The notification you mentioned is around the realtime socket. You would expect to see the ‘connected’ status… and perhaps on a phone or device with unstable connection see the ‘disconnected’ status sometimes.

I couldn’t see it in your second screenshot, but seeing the ‘Socket Status Unknown’ and the app stuck on the initial loading screen is definitely unexpected.

You mentioned that you bought a lot of three iPad 6th generation devices and those are giving you the same issues. I just wanted to confirm that it’s the same thing where you are the “Socket Status Unknown” and the grey blank screen with the loading spinner?

To be fair, all browsers on iOS leverage Safari’s engine under the hood. Even Chrome. It’s a restriction Apple places on iOS devices.

With “URL Bookmark” are you referring to using the Add to Homescreen option in iOS?

And in “Guided Access”, is that using regular Safari just with the app ‘locked’ using Guided Access?

The reason I ask is the Add to Homescreen approach is kind of like the Kiosk apps where it’s using Safari for rendering, but it’s like its own sandboxed app. But I’m curious if just native Safari is experiencing this too.

What about a simplified dashboard? I didn’t see the whole dashboard, but if any of those media tiles are MJPEG or refreshing images, I wonder if that could contribute to the memory issue.

And if a simplified dashboard doesn’t help, it would be could to at least try a test account so we can narrow down what’s going on.

Cory_Booth · June 13, 2024, 4:36am

Yeah I got a bad shot - but luckily for me, one of the iPads was acting up downstairs.
Here it is:

Yep, exact same issues

Correct - similar to the instructions post on the SharpTools user guide for iPads.

Let me add a bit more:

I have literally removed every single app that is removable (even all the Apple Apps).
The only apps left are Kiosker Apps, Camera, Safari, and Messenger (but Messenger is shut down as well)

I have shut off iCloud syncing - absolutely nothing syncs to iCloud
I have shut off Siri entirely,
I have shut off background App refresh (except for Kiosker Pro)…
I have shut off Bluetooth
I have tried associating the WiFi to a dedicated AP (I have six APs in the house)
Notifications are off
I have one iPad with all Network protection shut off

Nothing seems to help

Cory_Booth · June 13, 2024, 4:40am

To answer your timeline issue…

I say about a year ago… Sorry I can’t be more specific…

Cory_Booth · June 13, 2024, 4:43am

As for the Bookmark method - I’ll explore that a bit more again.
The problem I found was, it had its own quirks when left onscreen for days on end.
I’d come to a dashboard sitting on the homescreen or unresponsive to touch…

I thought I read in some of the threads here the toast message “Socket connected” was a SharpTools thing? Is this correct?

Cory_Booth · June 13, 2024, 5:00am

I am going to start removing some of my widgets - starting first with Javascript then moving to animated GIFs and then see where it goes…

Cory_Booth · June 13, 2024, 4:50pm

As I troubleshoot this with folks from Kiosker Pro.

Thought I’d share some of the logs I see which may - or may not - provide any insights.

First, what I have changed (beyond where I was a year ago)…
I have upped all my refreshing images to no less than five minute refresh. Some I bumped to an hour refresh (all weather related). Internally, my cameras by IP - I bumped those to 15 seconds.

I removed a Javascript web part from WillyWeather (which provided a Weather Alert ticker type of display)…

Here are some of the log entries I found interesting:
2024-06-11 20:07:12 INFO - [“[SocketManager::PingUtil::ping] Performing async ping with pingId”, “ptYpej”]
2024-06-11 20:07:16 VERBOSE - [“Received 'collection.set' for 'publicTokens' with 1 documents”]
2024-06-11 20:07:12 ERROR - [“[WorkerHeartbeat] Heartbeat and ping failed. Attempting recreation. “]
2024-06-11 20:07:12 DEBUG - [”[SocketManager::PingUtil::awaitPong] PongTimeout”, “ptYpej”]
2024-06-11 20:07:12 VERBOSE - [“[SocketManager] SocketWorker is created.”]
2024-06-11 20:07:15 INFO - [“[SocketManager::success]”, “Initialized.”]
2024-06-11 20:07:15 INFO - [“[SocketManager::success]”, “Successfully sent command connect”]
2024-06-11 20:07:15 DEBUG - [“[SocketManager::socket] status=connecting”]

2024-06-11 19:41:06 DEBUG - [“Subscriptions are filtered from 241 to 148 based on device authorization for location - xxxxxxx”]
2024-06-11 19:41:06 DEBUG - [“Retrieving subscriptions from samsung-smartthings for location xxxxxxx”]
2024-06-11 19:41:06 DEBUG - [“There are 241 subscriptions in the database for location xxxxxx”]
2024-06-11 19:41:06 WARNING - ["Missing 11 subscriptions for location

----------FROM THE APP LOGS----------------

2024-06-13 11:08:43 INFO updateUIView(:context:): 46 - Screensaver WebView Representable updateUIView
2024-06-13 11:08:44 WARNING webViewWebContentProcessDidTerminate(:): 430 - WebView content process did terminate

josh · June 13, 2024, 5:34pm

Thanks for the updates. It seems like the pages may be “crashing” in some way since you’re seeing the grey blank screen with “Socket status unknown”.

Apps vs Dashboard Content
I would be less concerned about the other apps on the iPad and more curious about the content on the dashboard itself which is why I was suggesting paring down the dashboard or even setting up a simplified test account.

Releases over Past Year
A year ago is quite a while back. The announcements category sorted by created date gives an idea of the items released over that time.

Starting around June '23 was the release of the Homey Integration but that was mostly a backend change. There was a bunch of other improvements including Faster Custom Tiles, Autorefreshing Backgrounds, Rule Engine Improvements, Hero Attribute Improvements, and more, but I can’t think of anything that would contribute to this type of issue.

Sockets & Background Worker
The realtime sockets in SharpTools weren’t released to production until February '24. In November '23, we released an early implementation of the Background Worker, but it wasn’t using a realtime socket for pushing out data – we end up refining that and rolling it out as a background thing in late '23 with the intention being that it would allow us to push out a message to devices running old versions of the web-app when we later did the realtime socket release several months later.

Log Entries
The log entries from the SharpTools side of things are mostly normal - receiving documents, periodic heartbeats, and the socket (re)connecting are normal.

While the healthcheck on the background worker is intentional and designed to recreate the worker in the rare event that it was terminated, it’s not something I would expect to commonly see in day-to-day usage. So that’s interesting that it needed to recreate itself.

Cory_Booth · June 13, 2024, 5:49pm

Thank you for your always responsiveness and assistance.

Ultimately my dashboard has been fairly static since I originally bragged about it in the “Show Me Your Dashaboards” threads.

On the homepage, I have three static JPGs and two animated GIFs. I am using your OpenWeather Title and then I fill the rest in with “buttons” - none are Hero or Super… But highly customized in terms of Images/Colors

I am testing the idea of slowly removing a few blocks one at a time and waiting a period of time…

I will keep you informed

Cory_Booth · June 21, 2024, 1:53pm

Circling back on this…

As I mentioned, for the most part my dashboards are pretty simple.
I have a lot of devices, but ultimately I just have several pages of on/off buttons.

My Home page does have several weather related tiles. - couple static JPGs and a couple animated GIFs.

One special script I have is related to weather alerts.
When there is a weather alert, a red banner will appear in the tile.
Otherwise, the tile is empty.

It does suck up some tile space, but I made it work by adding it to the bottom of my pages.

This script is available here:

I removed this script from my dashboards and it would seem this alleviated the majority of problems I was seeing.

I did have one dashboard time out “Socket Status Unknown” but this was after three days of no issues (which I haven’t seen in months)…

But this has definitely caused me to rethink where I should be looking for the actual problem.

First I thought it was iOS and the newer versions killing the older iPads.
Then I thought it may be related to Apples built in apps (Siri / Background App Refresh / Bluetooth / etc…)
Then I thought it may be the app I was using (Kiosker Pro)
Then I thought it may be the hardware (out of memory - I had cheap 16GB Gen 2s)

But now I can see it may simply be the code on the “website” that is added via this script or perhaps even other widgets.

Script is included below - ultimately it’s pretty simple

(function() {
    var isLoaded = false;
    var testWarning = document.location.href.indexOf('testWarning=1') !== -1;
    var loadWidget = function() {
        if (!isLoaded) {
            isLoaded = true;
            var url = 'https://cdnres.willyweather.com/widget/warning/dataView.html?id=151888';
            url = testWarning ? url + '&testWarning=1' : url;

            var div = document.createElement('div');
                        
            div.innerHTML = '<div id="ww-widget-warning" style="position: fixed; bottom: 0; right: 0; left: 0; width: 100%; min-height: 42px; z-index: 2147483647; border: none; overflow: hidden;">' +
                                '<button id="ww-widget-warning-button" style="color: #fff !important; background: rgba(255, 255, 255, .3) !important;display: inline-block !important; position: absolute !important; top: 9px !important; right: 6px !important; width: 24px !important; height:24px !important; border: none !important; border-radius: 50% !important; cursor: pointer !important; box-sizing: border-box !important; font-size: 17px !important; font-weight: normal !important; padding: 2px 0px 4px !important; line-height: 1 !important; vertical-align: middle !important; user-select: none !important; text-align: center !important; z-index: 2147483647 !important; font-family: sans-serif !important;">&times;</button>' +
                                '<iframe src="' + url + '" style="position: absolute !important; top: 0 !important; bottom: 0 !important; right: 0 !important; left: 0!important; width: 100% !important; height: 100% !important; z-index: 2147483646 !important; border: none !important; margin: 0 !important; padding: 0 !important; min-width: auto !important;"></iframe>' +
                            '</div>';
            document.body.appendChild(div);

                                        /* Responsive */
                var wwResizeWidget = function() {
                    var widgetEl = document.getElementById('ww-widget-warning');
                    if(window.innerWidth <= 600) {
                        widgetEl.style.minHeight = '72px';
                    } else {
                        widgetEl.style.minHeight = '42px';
                    }
                };

                window.addEventListener('resize', wwResizeWidget, false);
                wwResizeWidget();
            
            /* close button */
            document.getElementById('ww-widget-warning-button').onclick = function() {
                var warningWidget = document.getElementById('ww-widget-warning');
                warningWidget.parentNode.removeChild(warningWidget);
                isLoaded = false;
            }
        }
    };

    if (0 > 0 || document.location.href.indexOf('testWarning=1') !== -1) {
        window.addEventListener('load', loadWidget, false);
    }

    var keyup = function(e) {
        if (e.altKey && e.ctrlKey && (e.keyCode === 84 || e.which === 84)) {
            testWarning = true;
            loadWidget();
        }
    }
    window.addEventListener('keyup', keyup, false);
})();

josh · June 24, 2024, 7:23pm

Thanks for the update. That is interesting that the weather script seems to be contributing to the issues.

There’s a few other weather data related threads in the community that you might find interesting. For example, some community members have setup SharpTools Rules to pull weather information from Open Weather Map and they store things like weather warnings in SharpTools Variables.