tag:status.serenytics.com,2005:/historySerenytics Status - Incident History2024-03-04T10:17:24+01:00Serenyticstag:status.serenytics.com,2005:Incident/201045292024-02-29T04:00:00+01:002024-02-29T08:39:13+01:00Performance degraded on jobs execution<p><small>Feb <var data-var='date'>29</var>, <var data-var='time'>04:00</var> CET</small><br><strong>Resolved</strong> - The performances of the jobs execution was slower than usual this night due to a server issue.<br />Jobs were executed but possibility with some hours of delay.<br />Everything is back to normal now.</p>tag:status.serenytics.com,2005:Incident/169716942023-04-26T17:55:09+02:002023-04-26T17:56:10+02:00Queuing incident<p><small>Apr <var data-var='date'>26</var>, <var data-var='time'>17:55</var> CEST</small><br><strong>Resolved</strong> - This incident has been resolved. We apologize if this has created issues on your side.<br />The origin of the incident was a configuration issue in our architecture engine (i.e. on a replica set of a mongo db). We have identified the origin of the incident and have taken actions so it does not happen again.</p><p><small>Apr <var data-var='date'>26</var>, <var data-var='time'>16:35</var> CEST</small><br><strong>Monitoring</strong> - We have fixed the issue and are currently monitoring the platform.</p><p><small>Apr <var data-var='date'>26</var>, <var data-var='time'>14:58</var> CEST</small><br><strong>Investigating</strong> - All data queries are currently blocked. We are currently investigating.</p>tag:status.serenytics.com,2005:Incident/157554672023-01-02T10:47:34+01:002023-01-02T10:47:34+01:00instability of one automation server<p><small>Jan <var data-var='date'> 2</var>, <var data-var='time'>10:47</var> CET</small><br><strong>Resolved</strong> - This incident has been resolved.</p><p><small>Jan <var data-var='date'> 2</var>, <var data-var='time'>06:36</var> CET</small><br><strong>Investigating</strong> - One server executing automations was instable this morning. It has been restarted and the system is back to a normal state. Because of the reboot, some automation which started at 6am may have failed.</p>tag:status.serenytics.com,2005:Incident/150685642022-12-13T10:11:41+01:002022-12-13T10:11:41+01:00Scripts delay<p><small>Dec <var data-var='date'>13</var>, <var data-var='time'>10:11</var> CET</small><br><strong>Resolved</strong> - The platform is back to a normal state. All the automations which had been delayed have been executed.</p><p><small>Dec <var data-var='date'>13</var>, <var data-var='time'>09:32</var> CET</small><br><strong>Investigating</strong> - We've had one server down this morning and it causes delays in automations' execution. The issue is fixed but this will cause a delay in your script execution this morning.</p>tag:status.serenytics.com,2005:Incident/87132642021-11-29T09:39:25+01:002021-11-29T09:39:25+01:00Slow-down in automations execution<p><small>Nov <var data-var='date'>29</var>, <var data-var='time'>09:39</var> CET</small><br><strong>Resolved</strong> - This incident has been resolved.</p><p><small>Nov <var data-var='date'>29</var>, <var data-var='time'>09:23</var> CET</small><br><strong>Identified</strong> - We have a slow-down on automations execution. One of our machine is down causing this slow-down. All your scripts will be executed but they may be delayed.</p>tag:status.serenytics.com,2005:Incident/79280382021-09-06T11:47:08+02:002021-09-06T11:47:08+02:00Issues with one server executing automations<p><small>Sep <var data-var='date'> 6</var>, <var data-var='time'>11:47</var> CEST</small><br><strong>Resolved</strong> - This incident has been resolved. One of our server (dedicated to automation execution) was unreachable since Sunday morning (7:18am). The automations started on this machine failed. This server should have restarted automatically but it didn't. We need to fix this issue.<br /><br />Meanwhile, our load-balancing system spread the work on other machines and most of the automation jobs did run. But this Monday morning, as there were too many automation jobs for a reduced number of servers, the automations may have been delayed for a few hours.</p><p><small>Sep <var data-var='date'> 6</var>, <var data-var='time'>10:01</var> CEST</small><br><strong>Update</strong> - We are continuing to investigate this issue.</p><p><small>Sep <var data-var='date'> 6</var>, <var data-var='time'>09:59</var> CEST</small><br><strong>Investigating</strong> - We are currently investigating this issue.</p>tag:status.serenytics.com,2005:Incident/75937182021-07-24T18:53:57+02:002021-07-24T19:02:13+02:00Issue in our query engine<p><small>Jul <var data-var='date'>24</var>, <var data-var='time'>18:53</var> CEST</small><br><strong>Resolved</strong> - We've had an issue in the system we use to handle the list of data queries. It started at 6:15pm french time, and was resolved at 6:47pm french time. During this time range, if your data was not in cache, your dashboards were displaying infinite spinners but not data could be displayed. And your automation were not started. We're sorry for the inconvenience. <br /><br />We have identified the origin of the issue and we'll deploy a fix in the coming weeks so it does not happen again.</p>tag:status.serenytics.com,2005:Incident/68691642021-04-28T08:44:00+02:002021-04-29T08:46:45+02:00Issues to display some KPI widgets in the Viewer app and when generating a PDF from a dashboard<p><small>Apr <var data-var='date'>28</var>, <var data-var='time'>08:44</var> CEST</small><br><strong>Resolved</strong> - Some KPI widgets were not displayed at all when a dashboard was opened in the Viewer app (even if it was ok in the studio app).<br />And the same problem generated blank pages when generating a PDF.<br /><br />We rollbacked to the previous version and are currently investigating/fixing the issue.</p>tag:status.serenytics.com,2005:Incident/66709962021-04-03T19:42:36+02:002021-04-03T19:42:36+02:00Issue on our SQL database<p><small>Apr <var data-var='date'> 3</var>, <var data-var='time'>19:42</var> CEST</small><br><strong>Resolved</strong> - This incident has been resolved.</p><p><small>Apr <var data-var='date'> 3</var>, <var data-var='time'>19:39</var> CEST</small><br><strong>Investigating</strong> - We are currently investigating.</p>tag:status.serenytics.com,2005:Incident/35442552020-02-10T08:57:08+01:002020-02-10T08:57:08+01:00Issues with one server causing slowdowns<p><small>Feb <var data-var='date'>10</var>, <var data-var='time'>08:57</var> CET</small><br><strong>Resolved</strong> - This incident has been resolved.</p><p><small>Feb <var data-var='date'>10</var>, <var data-var='time'>08:40</var> CET</small><br><strong>Investigating</strong> - We are currently investigating this issue.</p>tag:status.serenytics.com,2005:Incident/23639822019-05-02T19:38:58+02:002019-05-02T19:38:58+02:00Scripts issues<p><small>May <var data-var='date'> 2</var>, <var data-var='time'>19:38</var> CEST</small><br><strong>Resolved</strong> - This incident has been resolved.</p><p><small>May <var data-var='date'> 2</var>, <var data-var='time'>10:50</var> CEST</small><br><strong>Update</strong> - We are continuing to monitor for any further issues.</p><p><small>May <var data-var='date'> 2</var>, <var data-var='time'>10:48</var> CEST</small><br><strong>Monitoring</strong> - Our servers dedicated to run scripts (ETL steps, Python scripts...) had issues this morning. We have restarted our platform and scripts are executed again. But there is a two hours delay between a script scheduling and its execution. That will be resolved during the day as our servers execute all scheduled scripts.</p>tag:status.serenytics.com,2005:Incident/17258342018-05-11T15:51:08+02:002018-05-11T15:51:08+02:00Elevated API Errors - dashboards not displaying data<p><small>May <var data-var='date'>11</var>, <var data-var='time'>15:51</var> CEST</small><br><strong>Resolved</strong> - This incident has been resolved.</p><p><small>May <var data-var='date'>11</var>, <var data-var='time'>12:44</var> CEST</small><br><strong>Investigating</strong> - We've experienced an elevated level of API errors for two hours this morning following our maintenance last week, but eveything is back to normal.</p>tag:status.serenytics.com,2005:Incident/17130612018-05-05T12:38:22+02:002018-05-05T12:38:22+02:00Infrastructure update<p><small>May <var data-var='date'> 5</var>, <var data-var='time'>12:38</var> CEST</small><br><strong>Completed</strong> - The scheduled maintenance has been completed.</p><p><small>May <var data-var='date'> 5</var>, <var data-var='time'>10:30</var> CEST</small><br><strong>In progress</strong> - Scheduled maintenance is currently in progress. We will provide updates as necessary.</p><p><small>May <var data-var='date'> 2</var>, <var data-var='time'>11:24</var> CEST</small><br><strong>Scheduled</strong> - We are migrating our infrastructure this week-end. As a consequence, our internal data warehouse and our main database will be read only during the maintenance. Thus editing dashboards as well as reloading or editing data in storage data-sources won't be possible during the maintenance. We will keep you updated.</p>tag:status.serenytics.com,2005:Incident/15628712018-01-11T21:40:18+01:002018-01-11T21:40:18+01:00Elevated error rates during server upgrade<p><small>Jan <var data-var='date'>11</var>, <var data-var='time'>21:40</var> CET</small><br><strong>Resolved</strong> - We just experienced elevated API error rates while rebooting sequentially our nodes to install security updates. Incident is resolved.</p>tag:status.serenytics.com,2005:Incident/14269152017-10-20T22:57:45+02:002017-10-20T22:57:45+02:00Internal data warehouse maintenance<p><small>Oct <var data-var='date'>20</var>, <var data-var='time'>22:57</var> CEST</small><br><strong>Completed</strong> - The scheduled maintenance has been completed.</p><p><small>Oct <var data-var='date'>20</var>, <var data-var='time'>21:03</var> CEST</small><br><strong>Update</strong> - Average API response time metrics is disabled during the data warehouse maintenance. However the API is still fully operational.</p><p><small>Oct <var data-var='date'>20</var>, <var data-var='time'>20:30</var> CEST</small><br><strong>In progress</strong> - Scheduled maintenance is currently in progress. We will provide updates as necessary.</p><p><small>Oct <var data-var='date'>19</var>, <var data-var='time'>17:51</var> CEST</small><br><strong>Scheduled</strong> - Our internal data warehouse will be read-only during our maintenance this week-end. As a consequence, reloading or editing data in storage data-sources won't be available. Sorry for the inconvenience.</p>tag:status.serenytics.com,2005:Incident/13075732017-07-25T22:30:49+02:002017-07-25T22:33:06+02:00Elevated API Errors<p><small>Jul <var data-var='date'>25</var>, <var data-var='time'>22:30</var> CEST</small><br><strong>Resolved</strong> - Maintenance is over, everything is back to normal.</p><p><small>Jul <var data-var='date'>25</var>, <var data-var='time'>21:33</var> CEST</small><br><strong>Identified</strong> - We're experiencing an elevated level of API errors due to an ongoing maintenance.</p>