Differences between revisions 29 and 427 (spanning 398 versions)
Revision 29 as of 2010-12-01 07:34:16
Size: 3100
Editor: bonaccos
Comment:
Revision 427 as of 2017-03-27 08:23:28
Size: 3682
Editor: bonaccos
Comment:
Deletions are marked like this. Additions are marked like this.
Line 1: Line 1:
<<Anchor(2010-11-26-servers-down)>> = General Informations =
 * This page lists announcements and status messages for IT services managed by [[http://www.isg.ee.ethz.ch/|ISG.EE]].
 * For notifications and announcements of central IT services managed by ID, please visit https://www1.ethz.ch/id/servicedesk/sysstat/index_EN
 * For a detailed status overview of central IT services managed by ID, please visit http://eranger3.ethz.ch/Ueberwachung/index.html
Line 3: Line 6:
= cooling water system outage on clusters = ||||<style="border-width: 1px 0px; border-color: rgb(85, 136, 238); padding: 0.6em;">'''Status-Key''' ||
||<style="border: medium none;"> {{attachment:Status/green.gif}} ||<style="border: medium none;">Resolved ||
||<style="border: medium none;"> {{attachment:Status/orange.gif}} ||<style="border: medium none;">Still working but with some errors ||
||<style="border-width: medium medium 1px; border-top: medium none rgb(85, 136, 238); border-left: medium none rgb(85, 136, 238); border-right: medium none rgb(85, 136, 238); border-color: rgb(85, 136, 238);"> {{attachment:Status/red.gif}} ||<style="border-width: medium medium 1px; border-top: medium none rgb(85, 136, 238); border-left: medium none rgb(85, 136, 238); border-right: medium none rgb(85, 136, 238); border-color: rgb(85, 136, 238);">Pending ||
Line 5: Line 11:
'''2010-11-26: 5:00 PM''' = Current status reports =
Line 7: Line 13:
Host autserv02 is running as well. All hosts can be used. <<Anchor(2017-03-24-cronbox-login-ssh-keys)>>
Line 9: Line 15:
'''2010-11-26: 4:40 PM''' == Cronbox/Login Server migration: new SSH host key ==
'''Status:''' {{attachment:Status/green.gif}}
Line 11: Line 18:
Server racks are cooled again, all hosts except of autserv02 are running and can be used.

'''2010-11-26: 4:00 PM'''

Server racks are still down.
--> Update follows at 5 PM or earlier

'''2010-11-26: 3:10 PM'''

One of the cooling water pumps installed in ETZ/D/96.2 does not work correctly. This forces some of the racks in this server room to shutdown
in order to protect the servers from thermal damage. '''clusters from IFH, IBT, BIWI, TIK, IKT and VAW are affected.'''
the facility management is working on solving the problem.
--> Update follows at 4 PM
  2017-03-24 17:00:: The cronbox and login server has moved to a new host. A new SSH host key has been generated:
  {{{
RSA fc:a8:00:5b:64:90:86:a1:fb:49:75:ef:55:58:90:b3
}}}
 Remember: '''Always''' verify a fingerprint of a SSH host key before accepting it.
Line 27: Line 26:
<<Anchor(2010-11-23-email-phishing-attack)>> <<Anchor(2017-01-07-Mailsystem migration)>>
Line 29: Line 28:
= Email Phishing attack = == EE Mailsystem migration ==
'''STATUS:''' {{attachment:Status/green.gif}} '''Mailsystem up'''
Line 31: Line 31:
'''2010-11-23'''   2017-01-08 15:00:: The new mailsystem is now started. In case of unattended problems we will stop it again to prevent data loss and to analyze the problem.
Line 33: Line 33:
Yesterday between 18:20 and 19:50 about 320 Phishing Mails have been sent to different Users at D-ITET. The Mails pretend to come from ''IT Support Group'' and contain the subject ''ISG.EE Webmail Alert''. The mail tells something about ''spammers'' that have compromised ''the'' ISG.EE Webmail Account and that you should provide your '''Username, Password''' and some '''Alternate Email'''. Please remember, that the ISG.EE Team will '''NEVER ask you for your Password!''' If you still have replied to this phishing mail please contact us '''immediately''' under support@ee.ethz.ch so that we can plan with you the next steps to keep your account safe.   2017-01-07 24:00:: Not all testcases could be performed. We now plan to enable the new system about noon.

  2017-01-07 20:45:: Old Mailserver Configuration migrated, starting the mailserver testing

  2017-01-07 14:00:: User mailbox data migrated, starting mailserver configuration migration

  2017-01-07 07:00:: All mail services are stopped. Mailbox data copy started.
Line 37: Line 43:
<<Anchor(2010-11-17-oenone-crash)>> <<Anchor(2016-09-12-network-outage)>>
Line 39: Line 45:
= oenone home server crash = == Networkoutage ETH ==
'''STATUS:''' {{attachment:Status/green.gif}}
Line 41: Line 48:
'''2010-11-17'''   2016-02-09 08:20:: ETH wide network outage due to hardware problems for the firewall infrastructure. In any case, please reboot your computer before continue.
Line 43: Line 50:
During this night at around 00:15 '''oenone''' one of our home-servers crashed. Users with homes on oenone where affected, these are '''BIWI''', '''VAW''', '''Collegium Helveticum''', '''Control''', '''IBT''', '''IKT'''. The server is now checking the filesystems and comming up again.   2016-02-09 12:35:: Network is back online and services are being recovered. Due to the hardware failure 53 network zones were affected. The problem got localized and resolved.
Line 45: Line 52:
We are sorry for the caused inconvenience and we are investigating the problem.

'''Update: 08:00''': oenone is now up and running again.

'''Update: 2010-11-18 07:30''' We opened a support case at Sun/Oracle for this server.
  2016-02-09 14:25:: Our systems should be all back to normal. In case you experience any problem please contact support via mailto:support@ee.ethz.ch.
Line 53: Line 56:
<<Anchor(2010-11-16-servers-down)>> <<Anchor(2016-02-10-maintenance-polaris)>>
Line 55: Line 58:
= cooling water system outage for some clusters = == Maintenance login.ee.ethz.ch and cronbox.ee.ethz.ch service ==
'''STATUS:''' {{attachment:Status/green.gif}}
Line 57: Line 61:
'''2010-11-16:'''   2016-02-10: 06:05:: The server for the [[Services/Cronjob|cronbox]] and login service is currently beeing updated from Debian Wheezy to Debian Jessie. The services will be temporarly unavailable.
Line 59: Line 63:
On last friday evening one of the cooling water pumps installed in ETZ/D/96.2 stopped working correctly. This forced some of the racks in this server room to shutdown in order to protect the servers from thermal damage. '''All clusters from IFH, IBT, BIWI, TIK, IKT and VAW were affected.'''   2016-02-10: 12:00:: Server update is done.
----
= Archived status reports =
Line 61: Line 67:

The facility management is working on solving the problem.

The servers are currently (08:35) down again. Please, even if they come up again, do not use them for long-timed computations as we still do not know when exactly the technician has solved the issue.

'''Update: 2010-11-17 08:25:''' The rack systems are running now with only one cooling water pump. A new pump is ordered by the rack company.

'''Update: 2010-11-18 16:00:''' Planed substitution of broken pump will be on 25.11 or 26.11.
[[Status/Archive/2015|2015]]
[[Status/Archive/2014|2014]]
[[Status/Archive/2013|2013]]
[[Status/Archive/2012|2012]]
[[Status/Archive/2011|2011]]
[[Status/Archive/2010|2010]]
Line 71: Line 75:
[[CategoryEDUC]]

General Informations

Status-Key

Status/green.gif

Resolved

Status/orange.gif

Still working but with some errors

Status/red.gif

Pending

Current status reports

Cronbox/Login Server migration: new SSH host key

Status: Status/green.gif

2017-03-24 17:00
The cronbox and login server has moved to a new host. A new SSH host key has been generated:
RSA fc:a8:00:5b:64:90:86:a1:fb:49:75:ef:55:58:90:b3
  • Remember: Always verify a fingerprint of a SSH host key before accepting it.


EE Mailsystem migration

STATUS: Status/green.gif Mailsystem up

2017-01-08 15:00
The new mailsystem is now started. In case of unattended problems we will stop it again to prevent data loss and to analyze the problem.
2017-01-07 24:00
Not all testcases could be performed. We now plan to enable the new system about noon.
2017-01-07 20:45
Old Mailserver Configuration migrated, starting the mailserver testing
2017-01-07 14:00
User mailbox data migrated, starting mailserver configuration migration
2017-01-07 07:00
All mail services are stopped. Mailbox data copy started.


Networkoutage ETH

STATUS: Status/green.gif

2016-02-09 08:20
ETH wide network outage due to hardware problems for the firewall infrastructure. In any case, please reboot your computer before continue.
2016-02-09 12:35
Network is back online and services are being recovered. Due to the hardware failure 53 network zones were affected. The problem got localized and resolved.
2016-02-09 14:25

Our systems should be all back to normal. In case you experience any problem please contact support via mailto:support@ee.ethz.ch.


Maintenance login.ee.ethz.ch and cronbox.ee.ethz.ch service

STATUS: Status/green.gif

2016-02-10: 06:05

The server for the cronbox and login service is currently beeing updated from Debian Wheezy to Debian Jessie. The services will be temporarly unavailable.

2016-02-10: 12:00
Server update is done.


Archived status reports

2015 2014 2013 2012 2011 2010


CategoryEDUC

Status (last edited 2023-10-16 11:24:17 by alders)