Differences between revisions 496 and 657 (spanning 161 versions)
Revision 496 as of 2018-06-16 03:46:25
Size: 10914
Editor: bonaccos
Comment:
Revision 657 as of 2022-10-24 10:34:36
Size: 791
Editor: mreimers
Comment:
Deletions are marked like this. Additions are marked like this.
Line 1: Line 1:
#rev 2018-08-27 mreimers
#rev 2020-08-31 alders
Line 2: Line 5:
 * This page lists announcements and status messages for IT services managed by [[http://www.isg.ee.ethz.ch/|ISG.EE]].
* /!\ Thes ISG.EE status page has been moved to https://status.isg.ee.ethz.ch
Line 6: Line 11:
||||<style="border-width: 1px 0px; border-color: rgb(85, 136, 238); padding: 0.6em;">'''Status-Key''' ||
||<style="border: medium none;"> {{attachment:Status/green.gif}} ||<style="border: medium none;">Resolved ||
||<style="border: medium none;"> {{attachment:Status/orange.gif}} ||<style="border: medium none;">Still working but with some errors ||
||<style="border-width: medium medium 1px; border-top: medium none rgb(85, 136, 238); border-left: medium none rgb(85, 136, 238); border-right: medium none rgb(85, 136, 238); border-color: rgb(85, 136, 238);"> {{attachment:Status/red.gif}} ||<style="border-width: medium medium 1px; border-top: medium none rgb(85, 136, 238); border-left: medium none rgb(85, 136, 238); border-right: medium none rgb(85, 136, 238); border-color: rgb(85, 136, 238);">Pending ||

= Current status reports =

<<Anchor(2018-06-16-D-ITET-mail-server-downtime)>>

== D-ITET mailserver downtime: New operating system version ==
'''Status:''' {{attachment:Status/red.gif}}

  2018-06-16 06:00:: Due to a planned operating system update, the D-ITET mail server will be unavailable today, June 16, 2018 between 06:00 and 08:00.

<<Anchor(2018-04-24-major-outage-incident)>>

== Major outage virtualization cluster/networking switch ==
'''Status:''' {{attachment:Status/green.gif}}

  2018-04-24 08:56:: Sending of emails is restored again. Recieving mail should not be lost for any properly sending email server, since the issues caused a temporary error notification to the sending server which should in turn retry resubmitting an email correctly later on with some delay.
  2018-04-24 07:45:: Bringing back online most important services, including home service; issue being investigated.
  2018-04-24 06:29:: Major outage of Networking/virtualization Cluster taking down important D-ITET Services (home Server, partially mailsystem, Linux clients).

<<Anchor(2018-04-06-jabba-maintenance)>>

== Jabba Maintenance ==
'''Status:''' {{attachment:Status/green.gif}}
  2018-04-06 08:10:: Jabba is back online
  2018-04-06 07:00:: Jabba is offline due to maintenance work

<<Anchor(2018-03-10-d-itet-storage-migration)>>

== D-ITET Storage Migration ==
'''Status:''' {{attachment:Status/green.gif}}
 
  2018-03-10 15:00:: Migration of user homes completed.
  2018-03-10 14:15:: User homes migrated, access is unblocked again, some post-migration tasks still pending.
  2018-03-10 10:00:: D-ITET user homes will be migrated from ID Storage to D-ITET Storage. During the whole migration time access to the user homes for the affected users is blocked. Affected users are informed directly by an email.

<<Anchor(2018-02-12-svnsrv-os-upgrade)>>
== svn.ee.ethz.ch Server migration: New operating system version ==
'''Status:''' {{attachment:Status/green.gif}}

  2018-02-12 08:55:: Server upgrade has been completed and all services up and running again.
  
  2018-02-12 06:15:: Start updating server from Debian Wheezy 7 to Debian Stretch 9. Downtimes for `https://svn.ee.ethz.ch`, `svn://svn.ee.ethz.ch` and `https://svnmgr.ee.ethz.ch`.

<<Anchor(2018-02-05-cronbox-os-upgrade)>>
== Cronbox/Login Server migration: New operating system version ==
'''Status:''' {{attachment:Status/green.gif}}

  2018-02-05 07:00:: The host `mira` has been upgraded to Debian 9 Stretch. SSH Host keys fingerprints for RSA and ED25519 are:
  {{{
4096 MD5:fc:a8:00:5b:64:90:86:a1:fb:49:75:ef:55:58:90:b3 (RSA)
4096 SHA256:v48HAAAjr+avnPAESdQzazSriKYZeTGGtIPKfoE8Dg0 (RSA)
256 SHA256:SgvaiZyIgzujLJdbtRij5VGUOXm/IuAs3MkMYtGZNhc (ED25519)
256 MD5:3b:b0:1a:8a:ea:0a:e5:ea:bb:9e:bb:5c:ef:24:c3:92 (ED25519)
}}}
The SSH host key is as well listed on: https://people.ee.ethz.ch/

  2018-01-31 11:00:: The host `mira` holding the cronbox and login service will be upgraded to Debian 9 Stretch on 2018-02-05 at 06:10.

<<Anchor(2018-01-25-upgrade-itetnas02)>>
== Upgrade of Server itetnas02 ==
'''Status:''' {{attachment:Status/green.gif}}

  2018-01-25 07:30:: Upgrade completed.

  2018-01-24 16:45:: On 2018-01-25 around 06:10 we will upgrade the server `itetnas02`. Several short outages for Fileservices (Samba, NFS) are expected. Services for project accounts and dedicated shares for biwi, ibt, ini and tik are affected.

<<Anchor(2017-11-10-outage-itetnas03)>>
== Outage of Server itetnas03 ==
'''Status:''' {{attachment:Status/green.gif}}
 
  2017-11-15 07:00:: Battery unit replaced

  2017-11-10 07:20:: Server is back online but without battery unit. We will need to shutdown `itetnas03` again once the problem is isolated and can be fixed.
 
  2017-11-10 06:15:: The server itetnas03 is down due to hardware problems (A battery replacement caused controller problems). ISG and the hardware vendor are currently working to get this problem solved.

<<Anchor(2017-11-07-CifsHome)>>
== User Home accessibility ==
'''Status:''' {{attachment:Status/green.gif}}

  2017-11-08 06:25:: Informatikdienste have reverted a change which caused the problems for accessing all user's HOME via the CIFS (SAMBA) protocol.

  2017-11-07 08:00:: All users' HOME are currently not accessible by CIFS (SAMBA) protocol. NFS access is still available.

<<Anchor(2017-10-24-ibtnas02)>>
== Outage of Server ibtnas02 ==
'''Status:''' {{attachment:Status/green.gif}}

  2017-10-31 08:00:: Upgrade successfully completed

  2017-10-30 16:50:: The server will be upgraded to a new OS release on 2017-10-31 starting around 06:15. Short outages of Samba and NFS services are going to be expected.

  2017-10-25 10:00:: ibtnas02 now serves all partitions but the problem is not yet identified

  2017-10-24 15:00:: The server ibtnas02 is up again (partition data-08 is not available)

  2017-10-24 12:50:: The server ibtnas02 is down again

  2017-10-24 09:30:: The server ibtnas02 is back online

  2017-10-24 08:00:: The server ibtnas02 is down due to hardware problems

<<Anchor(2017-10-21-itetnas03)>>
== Outage of Server itetnas03 ==
'''Status:''' {{attachment:Status/green.gif}}
  
  2017-10-23 18:15:: Data are also accessible via NFS.

  2017-10-23 9:30:: The server is up. Data are accessible via Samba. NFS file service is still down.

  2017-10-21 15:00:: The server itetnas03 is down due to hardware problems



<<Anchor(2017-10-18-outage-etz-d-96-2)>>
== Outage of Servers in Serverroom ETZ/D/96.2 ==
'''Status:''' {{attachment:Status/green.gif}}
  
  2017-10-20 13:45:: All racks in ETZ/D/96.2 are working again (cooling problem solved).

  2017-10-20 10:00:: The technician will arrive at 13:00 hours. Some servers are running, but without watercooling. So any rack might shutdown at any time if the air cooling is not sufficient. This will most probably again happen when the technician will be working in the room (i.e. this afternoon).

  2017-10-19 18:30:: The cooling engineer could not fix the problem, so some servers are still offline. Another technicial will try to fix the cooling system tomorrow morning.

  2017-10-18 14:00:: Cooling system is still not working correctly, we only selectively powered on a couple of compute machines.

  2017-10-18 12:50:: The problem has been localized and repaired. We need to wait that the circuit is cooling down.

  2017-10-18 10:30:: Outage of most racks in ETZ/D/96.2 (cooling problem) . Most compute servers are offline.


<<Anchor(2017-05-13-outage-etz-d-96-2)>>
== Outage Servers in Serverroom ETZ/D/96.2 ==
'''Status:''' {{attachment:Status/green.gif}}

  2017-05-13 20:00:: Outage of some racks in ETZ/D/96.2. Several compute servers offline.
  2017-05-13 23:59:: Most of the servers are back online.
  2017-05-15 08:45:: Status of remaining servers verified. All back online.

<<Anchor(2017-03-24-cronbox-login-ssh-keys)>>
== Cronbox/Login Server migration: new SSH host key ==
'''Status:''' {{attachment:Status/green.gif}}

  2017-03-24 17:00:: The cronbox and login server has moved to a new host. A new SSH host key has been generated:
  {{{
4096 MD5:fc:a8:00:5b:64:90:86:a1:fb:49:75:ef:55:58:90:b3 (RSA)
4096 SHA256:v48HAAAjr+avnPAESdQzazSriKYZeTGGtIPKfoE8Dg0 (RSA)
}}}
The SSH host key is as well listed on: https://people.ee.ethz.ch/

  Remember:: '''Always''' verify a fingerprint of a SSH host key before accepting it.

<<Anchor(2017-01-07-Mailsystem migration)>>
== EE Mailsystem migration ==
'''STATUS:''' {{attachment:Status/green.gif}} '''Mailsystem up'''

  2017-01-08 15:00:: The new mailsystem is now started. In case of unattended problems we will stop it again to prevent data loss and to analyze the problem.

  2017-01-07 24:00:: Not all testcases could be performed. We now plan to enable the new system about noon.

  2017-01-07 20:45:: Old Mailserver Configuration migrated, starting the mailserver testing

  2017-01-07 14:00:: User mailbox data migrated, starting mailserver configuration migration

  2017-01-07 07:00:: All mail services are stopped. Mailbox data copy started.

<<Anchor(2016-09-12-network-outage)>>
== Networkoutage ETH ==
'''STATUS:''' {{attachment:Status/green.gif}}

  2016-02-09 08:20:: ETH wide network outage due to hardware problems for the firewall infrastructure. In any case, please reboot your computer before continue.

  2016-02-09 12:35:: Network is back online and services are being recovered. Due to the hardware failure 53 network zones were affected. The problem got localized and resolved.

  2016-02-09 14:25:: Our systems should be all back to normal. In case you experience any problem please contact support via mailto:support@ee.ethz.ch.

<<Anchor(2016-02-10-maintenance-polaris)>>
== Maintenance login.ee.ethz.ch and cronbox.ee.ethz.ch service ==
'''STATUS:''' {{attachment:Status/green.gif}}

  2016-02-10: 06:05:: The server for the [[Services/Cronjob|cronbox]] and login service is currently beeing updated from Debian Wheezy to Debian Jessie. The services will be temporarly unavailable.

  2016-02-10: 12:00:: Server update is done.
Line 196: Line 14:
[[Status/Archive/2010|2010]]
[[Status/Archive/2011|2011]]
[[Status/Archive/2012|2012]]
[[Status/Archive/2013|2013]]
[[Status/Archive/2014|2014]]
Line 197: Line 20:
[[Status/Archive/2014|2014]]
[[Status/Archive/2013|2013]]
[[Status/Archive/2012|2012]]
[[Status/Archive/2011|2011]]
[[Status/Archive/2010|2010]]
[[Status/Archive/2016|2016]]
[[Status/Archive/2017|2017]]
[[Status/Archive/2018|2018]]
[[Status/Archive/2019|2019]]

General Informations

Archived status reports

2010 2011 2012 2013 2014 2015 2016 2017 2018 2019


CategoryEDUC

Status (last edited 2023-10-16 11:24:17 by alders)