|
Size: 8487
Comment:
|
Size: 10127
Comment:
|
| Deletions are marked like this. | Additions are marked like this. |
| Line 1: | Line 1: |
| #rev 2018-08-27 mreimers #rev 2020-08-31 alders = General Informations = * This page lists announcements and status messages for IT services managed by [[http://www.isg.ee.ethz.ch/|ISG.EE]]. * For notifications and announcements of central IT services managed by ID, please visit https://www.ethz.ch/services/de/it-services/service-desk.html * For a detailed status overview of central IT services managed by ID, please visit https://ueberwachung.ethz.ch |
|
| Line 2: | Line 10: |
| ||<style="border: medium none;"> {{attachment:green.gif}} ||<style="border: medium none;">Resolved || ||<style="border-width: medium medium 1px; border-top: medium none rgb(85, 136, 238); border-left: medium none rgb(85, 136, 238); border-right: medium none rgb(85, 136, 238); border-color: rgb(85, 136, 238);"> {{attachment:red.gif}} ||<style="border-width: medium medium 1px; border-top: medium none rgb(85, 136, 238); border-left: medium none rgb(85, 136, 238); border-right: medium none rgb(85, 136, 238); border-color: rgb(85, 136, 238);">Pending || |
||<style="border: medium none;"> {{attachment:Status/green.gif}} ||<style="border: medium none;">Resolved || ||<style="border: medium none;"> {{attachment:Status/orange.gif}} ||<style="border: medium none;">Still working but with some errors || ||<style="border-width: medium medium 1px; border-top: medium none rgb(85, 136, 238); border-left: medium none rgb(85, 136, 238); border-right: medium none rgb(85, 136, 238); border-color: rgb(85, 136, 238);"> {{attachment:Status/red.gif}} ||<style="border-width: medium medium 1px; border-top: medium none rgb(85, 136, 238); border-left: medium none rgb(85, 136, 238); border-right: medium none rgb(85, 136, 238); border-color: rgb(85, 136, 238);">Pending || |
| Line 5: | Line 14: |
| <<Anchor(2011-02-28-zhadum-crash)>> | = Current status reports = |
| Line 7: | Line 16: |
| = Windows Terminal Server zhadum out-of-operation = '''STATUS:''' {{attachment:red.gif}} |
<<Anchor(2022-07-16-planned-server-downtime)>> == Planned ISG.EE Mailsystem downtime because of IMAP server platform switch == '''Status:''' {{attachment:Status/green.gif}} |
| Line 10: | Line 20: |
| '''2011-02-28 14:00''' | 2022-07-16 11:00:: The update of our IMAP server to a new platform has completed successfully. The Mailsystem is available again. 2022-07-16 07:00:: Update of our IMAP server to a new platform has started. Therefore the mail system will not be available for our customers. 2022-07-16 07:00:: On 2022-07-16 between 07:00 and 12:00 ISG is going to migrate the IMAP Server to another system. During the migration there will be no access to the E-Mails for the following Domains: * ee.ethz.ch * vision.ee.ethz.ch * isi.ee.ethz.ch * mwe.ee.ethz.ch * mins.ee.ethz.ch * nari.ee.ethz.ch |
| Line 12: | Line 30: |
| The hardware of the departemental Windows terminal server '''zhadum''' is broken. A replacement server should be available soon... | <<Anchor(2022-07-09-netscratch-maintenance)>> == NetScratch Server Filesystem Maintenance == '''Status:''' {{attachment:Status/green.gif}} 2022-07-10 10:55:: Fileysstem for [[Services/NetScratch|NetScratch]] is online and in read-write mode. 2022-07-10 09:30:: First run of checks are completed and we are proceeding with the next steps to put the filesystem again online. 2022-07-09 13:00:: The [[Services/NetScratch|NetScratch]] filesystem will be put into read-only mode for maintenance. <<Anchor(2022-06-16-exchange-support)>> == Unreachable ee.ethz.ch Email recipients over ID Exchange Mailserver == '''Status:''' {{attachment:Status/green.gif}} 2022-06-16 16:45:: Configuration issue has been resolved. 2022-06-16 15:00:: Emails with ee.ethz.ch recipients sent over the ID Exchange Server do not reach destination. ID Exchange Admins are working on fixing the problem. <<Anchor(2021-11-01-home-maintenance)>> == HOME Server maintenance to repair fileystem inconsistence == '''Status:''' {{attachment:Status/green.gif}} 2021-11-01 06:30:: System back online and HOME directories are again accessible for all D-ITET user 2021-11-01 05:50:: HOME Server will be put offline to start a repair of a filesystem inconsistence <<Anchor(2021-07-05-linux-printing)>> == Linux printing affected by PrintNightmare vulnerability patch == '''Status:''' {{attachment:Status/green.gif}} 2021-07-05 13:00:: ID resolved the issue 2021-07-05 09:41:: Workaround: Use [[Printing#Platform-independent_printing| platform-independent printing]] 2021-07-05 09:41:: Authentification for printing fails. Ticket at ID servicedesk opened. <<Anchor(2021-04-27-itetmaster01-update)>> == Downtime various D-ITET services for server maintenance == '''Status:''' {{attachment:Status/green.gif}} 2021-04-27 08:30:: Condor is back online, all services restored. 2021-04-27 08:15:: Matrix/Element Chat services back online. 2021-04-27 08:00:: Database upgrade done and online. 2021-04-27 07:30:: Slurm services are back online. 2021-04-27 07:00:: Base system has been upgraded, main database services in progress of upgrade. 2021-04-27 06:00:: On 2021-04-27 between 06:00 and 08:30 ISG is going to update a server providing access to various D-ITET services. During the migration the following services will be affected and offline: * Matrix/Element Chat services (the instances will be unavailable) * IFA/Control Website: Access to the IFA database is blocked * Slurm (D-ITET Arton Cluster): It won't be possible to submit new jobs or view Slurm statistics. Already running jobs will not be affected. * Condor: Condor clients will be shut down the evening before to avoid running jobs during the migration. <<Anchor(2021-03-31-network-disruption)>> == Network disruption affecting several ISG.EE services == '''Status:''' {{attachment:Status/green.gif}} 2021-03-31 09:30:: The configuration error was found. The configuration change will be deployed on '''2021-04-01 around 06:15''' and a short network of about 1min is expected. 2021-03-31 08:00:: ID Networking team has rolled-back a deployed configuration, pending further investigation/analysis. 2021-03-31 07:30:: There are currently disruption affecting a VPZ with servers managed by ISG.EE. Networking team of ID is investigating the issue. There are several ISG.EE services affected/malfunctioning due to this in particluar the FindYourData service. <<Anchor(2021-03-11-mira-upgrade)>> == login.ee.ethz.ch: downtime for server upgrade == '''Status:''' {{attachment:Status/green.gif}} 2021-03-11 06:30:: Upgrade completed and service is up and running again. 2021-03-11 06:00:: The server servicing login.ee.ethz.ch will be upgraded to a new OS version (Debian buster). During the time of the update logins might not be possible. <<Anchor(2020-07-11-storage-downtime)>> == Planned project/ archive storage downtime and client reboot == '''Status:''' {{attachment:Status/green.gif}} 2020-07-11 12:00:: Migration has been completed, all services are back to operational state. 2020-07-11 08:00:: Migration started, services are shutdown 2020-07-11 8:00-12:00:: Start of planned maintenance work. Project/ archive storage services (known under the names "ocean", "bluebay", "lagoon" and "benderstor") will not be available. ISG-managed Linux clients will be rebooted. <<Anchor(2020-06-04-svnsrv-upgrade)>> == svn.ee.ethz.ch downtime for server upgrade == '''Status:''' {{attachment:Status/green.gif}} 2020-06-04 07:05:: Webservices for managing SVN repositories are enabled. 2020-06-04 06:15:: Systemupgrade is done and access to the SVN repositories via the `svn` and `https` transport protocols are back online. 2020-06-04 06:00:: The server servicing the SVN repositories will be upgraded to a new operating system version. During this timeframe outages for access to the SVN repositories are expected. <<Anchor(2020-05-17-cluster-abuse)>> == European HPC cluster abuse == '''Status:''' {{attachment:Status/green.gif}}<<BR>> Recently European HPC clusters have been attacked and abused for mining purposes. The D-ITET Slurm and SGE clusters have not been compromised. We are monitoring the situation closely. 2020-05 17 08:30:: No successful login from known attacker IP addresses could be determined, none of the files indicating being compromised have been found on our file systems 2020-05-16 14:30:: No unusal cluster job activity was observed <<Anchor(2020-05-04-itetnas04-upgrade)>> == D-ITET Netscratch downtime for server upgrade == '''Status:''' {{attachment:Status/green.gif}} 2020-05-04 06:00:: Server upgrade has been completed. 2020-05-04 06:00:: The server servicing the D-ITET Netscratch service will be upgraded to a new operating system version. During this timeframe outages for the NFS service will be expected. <<Anchor(2020-04-07-network-interuption)>> == Network outage ETx router == '''Status:''' {{attachment:Status/green.gif}} 2020-04-07 05:30:: There was an issue on the Router `rou-etx`. ID networking team trackled and solved the issue. There was about a 10min interuption for the ETx networking zone affecting almost all ISG.EE maintained systems. <<Anchor(2020-04-06-mira-maintenance)>> == login.ee.ethz.ch: Reboot for maintenance == '''Status:''' {{attachment:Status/green.gif}} 2020-04-06 05:35:: System behind `login.ee.ethz.ch` has been rebootet for maintenance and increase available resources. See the [[RemoteAccess|information on access D-ITET resources remotely]]. To distribute better the load user are encouraged to use the VPN service whenever possible. <<Anchor(2020-02-18-nostro-maintenance)>> == itet-stor (FindYourData) Server maintenance: Reconfiguration of VM parameters == '''Status:''' {{attachment:Status/green.gif}} 2020-02-18 19:03:: System again up and running. 2020-02-18 19:00:: Scheduled downtime for the [[Workstations/FindYourData|itet-stor/FindYourData service]] due to maintenance work on the underlying server. <<Anchor(2020-01-20-nostro-os-upgrade)>> == itet-stor (FindYourData) Server migration: New operating system version == '''Status:''' {{attachment:Status/green.gif}} 2020-01-20 07:15:: OS upgrade done, there were short interruptions to the [[Workstations/FindYourData|itet-stor/FindYourData service]]. 2020-01-20 06:00:: We will update the server servicing the [[Workstations/FindYourData|FindYourData service]] from Debian jessie 8 to Debian stretch 9. There will be short downtimes accessing this service during the update. = Archived status reports = [[Status/Archive/2010|2010]] [[Status/Archive/2011|2011]] [[Status/Archive/2012|2012]] [[Status/Archive/2013|2013]] [[Status/Archive/2014|2014]] [[Status/Archive/2015|2015]] [[Status/Archive/2016|2016]] [[Status/Archive/2017|2017]] [[Status/Archive/2018|2018]] [[Status/Archive/2019|2019]] |
| Line 15: | Line 164: |
| <<Anchor(2011-02-22-jabba-upgrade)>> = Upgrade of Backup Server JABBA = '''STATUS:''' {{attachment:green.gif}} '''2011-02-22 18:15 PM''' The migration is finished and the new JABBA server is online. '''2011-02-22 7:00 AM - 7:00 PM (approx.)''' We are going to upgrade the departments backup server JABBA. This upgrade includes changes in software as well as in hardware. During the upgrade no restore request of lost data can be fulfilled. '''The complete backup infrastructure and all belonging services are NOT available during the upgrade''' ---- <<Anchor(2011-02-07-agilent-license-server-change)>> = Agilent ADS/ICCAP License Server Change = '''STATUS:''' {{attachment:green.gif}} '''2011-02-07 10:00''' As announced a week ago, the license server for Agilent ADS and ICCAP software has changed. If your client software still uses the old license server information, please make sure you change the license server to '''lic-agilent.ee.ethz.ch'''. The port stays the same. ---- <<Anchor(2011-01-31-svn-firewall)>> = Subversion server not reachable from outside ETH Zurich = '''STATUS:''' {{attachment:green.gif}} '''2011-01-31 16:30''' Our subversion server is now available again for users outside of ETH Zurich. '''2011-01-31: 16:15''' We have been told that the central firewall rules cannot be changed right now due to other (completely unrelated) problems. At the moment it is unknown when this will be fixed. '''2011-01-31: 15:30''' Our subversion server svn.ee.ethz.ch is at the moment not accessible through the svn:// protocol from outside ETH Zurich due to a firewall configuration error. We are working on it... ---- <<Anchor(2011-01-26-mailser-problems-after-patching)>> = Mailserver problems while patching = '''STATUS:''' {{attachment:green.gif}} '''2011-01-26: 11:00 AM''' During the server upgrade last night a patch has temporarily misconfigured the mail server. The server accepted incoming mails but could not place them into the users mailbox. EVERY such mail created a bounce. Because of this, our statement that no mails get lost while updating the servers, is not fully true anymore. Incoming Mails which bounced back have to be resent by the sender. ---- <<Anchor(2010-12-14-solaris-server-patching)>> = Solaris Server Patching = '''STATUS:''' {{attachment:green.gif}} '''2011-01-25: 10:30 PM''' All servers and services are back online '''2011-01-25: 7:00 PM - 10:00 PM''' To keep our systems up to date with the newest software and security releases, we need to update our servers on a regular base. For this reason we are going to patch and reboot some of our Solaris servers. Servers concerned: '''drwho''', '''tardis''', '''oenone''', '''spitfire''', '''yosemite''', '''malina'''. To prevent data corruption/loss please do the following: * All diskless clients (Linux): please logout and shutdown all DL clients * All Windows systems: please logout and shutdown all Windows clients * All Mac systems: please logout and shutdown all Mac clients * All user homes: please logout from these servers * No NFS or SAMBA access to user homes * No mail access, no outgoing mails (incoming mails WON'T get lost) * Webpages hosted on these systems are unavailable ---- <<Anchor(2010-12-19-delayed-email-delivery)>> = Delayed Email delivery = '''STATUS:''' {{attachment:green.gif}} '''2011-01-19: 11:00 AM - 4:30 PM''' As a result of a faulty [[http://lurker.clamav.net/thread/20110119.125839.2b4ce0e1.en.html|ClamAV signature File]] every Email that contained a PDF-file was marked as infected. Before we could resend the quarantined emails we had to fix the issue. No mail was lost and everything was resent. Update: 2011-01-20: 10:33 AM:: ClamAV Signatues have been updated and tested. Everything is working as it should. ---- <<Anchor(2010-12-07-reboot-yosemite)>> = Maintenance Reboot of Solaris Server Yosemite = '''STATUS:''' {{attachment:green.gif}} '''2010-12-07: 7:30 AM''' Server yosemite has been rebooted successfully. All services are available. '''2010-12-07: 7:00 AM''' Due to a shortage of available memory we are forced to reboot the solaris server yosemite. Downtime approx. 30 minutes. ---- <<Anchor(2010-11-26-servers-down)>> = cooling water system outage on clusters = '''STATUS:''' {{attachment:green.gif}} '''2010-11-26: 5:00 PM''' Host autserv02 is running as well. All hosts can be used. '''2010-11-26: 4:40 PM''' Server racks are cooled again, all hosts except of autserv02 are running and can be used. '''2010-11-26: 4:00 PM''' Server racks are still down. --> Update follows at 5 PM or earlier '''2010-11-26: 3:10 PM''' One of the cooling water pumps installed in ETZ/D/96.2 does not work correctly. This forces some of the racks in this server room to shutdown in order to protect the servers from thermal damage. '''clusters from IFH, IBT, BIWI, TIK, IKT and VAW are affected.''' the facility management is working on solving the problem. --> Update follows at 4 PM ---- <<Anchor(2010-11-23-email-phishing-attack)>> = email phishing attack = '''STATUS:''' {{attachment:green.gif}} '''2010-11-23''' Yesterday between 18:20 and 19:50 about 320 Phishing Mails have been sent to different Users at D-ITET. The Mails pretend to come from ''IT Support Group'' and contain the subject ''ISG.EE Webmail Alert''. The mail tells something about ''spammers'' that have compromised ''the'' ISG.EE Webmail Account and that you should provide your '''Username, Password''' and some '''Alternate Email'''. Please remember, that the ISG.EE Team will '''NEVER ask you for your Password!''' If you still have replied to this phishing mail please contact us '''immediately''' under support@ee.ethz.ch so that we can plan with you the next steps to keep your account safe. ---- <<Anchor(2010-11-17-oenone-crash)>> = oenone home server crash = '''STATUS:''' {{attachment:green.gif}} '''2010-11-17''' During this night at around 00:15 '''oenone''' one of our home-servers crashed. Users with homes on oenone where affected, these are '''BIWI''', '''VAW''', '''Collegium Helveticum''', '''Control''', '''IBT''', '''IKT'''. The server is now checking the filesystems and comming up again. We are sorry for the caused inconvenience and we are investigating the problem. Update: 08:00:: oenone is now up and running again. Update: 2010-11-18 07:30:: We opened a support case at Sun/Oracle for this server. ---- <<Anchor(2010-11-16-servers-down)>> = cooling water system outage for some clusters = '''STATUS:''' {{attachment:green.gif}} '''2010-11-16:''' On last friday evening one of the cooling water pumps installed in ETZ/D/96.2 stopped working correctly. This forced some of the racks in this server room to shutdown in order to protect the servers from thermal damage. '''All clusters from IFH, IBT, BIWI, TIK, IKT and VAW were affected.''' The facility management is working on solving the problem. The servers are currently (08:35) down again. Please, even if they come up again, do not use them for long-timed computations as we still do not know when exactly the technician has solved the issue. Update: 2010-11-17 08:25:: The rack systems are running now with only one cooling water pump. A new pump is ordered by the rack company. Update: 2010-11-18 16:00:: Planed substitution of broken pump will be on 25.11 or 26.11. ---- |
[[CategoryEDUC]] |
General Informations
This page lists announcements and status messages for IT services managed by ISG.EE.
For notifications and announcements of central IT services managed by ID, please visit https://www.ethz.ch/services/de/it-services/service-desk.html
For a detailed status overview of central IT services managed by ID, please visit https://ueberwachung.ethz.ch
Status-Key |
|
|
Resolved |
|
Still working but with some errors |
|
Pending |
Current status reports
Planned ISG.EE Mailsystem downtime because of IMAP server platform switch
Status: 
- 2022-07-16 11:00
- The update of our IMAP server to a new platform has completed successfully. The Mailsystem is available again.
- 2022-07-16 07:00
- Update of our IMAP server to a new platform has started. Therefore the mail system will not be available for our customers.
- 2022-07-16 07:00
- On 2022-07-16 between 07:00 and 12:00 ISG is going to migrate the IMAP Server to another system. During the migration there will be no access to the E-Mails for the following Domains:
- ee.ethz.ch
- vision.ee.ethz.ch
- isi.ee.ethz.ch
- mwe.ee.ethz.ch
- mins.ee.ethz.ch
- nari.ee.ethz.ch
NetScratch Server Filesystem Maintenance
Status:
- 2022-07-10 10:55
Fileysstem for NetScratch is online and in read-write mode.
- 2022-07-10 09:30
- First run of checks are completed and we are proceeding with the next steps to put the filesystem again online.
- 2022-07-09 13:00
The NetScratch filesystem will be put into read-only mode for maintenance.
Unreachable ee.ethz.ch Email recipients over ID Exchange Mailserver
Status:
- 2022-06-16 16:45
- Configuration issue has been resolved.
- 2022-06-16 15:00
- Emails with ee.ethz.ch recipients sent over the ID Exchange Server do not reach destination. ID Exchange Admins are working on fixing the problem.
HOME Server maintenance to repair fileystem inconsistence
Status:
- 2021-11-01 06:30
- System back online and HOME directories are again accessible for all D-ITET user
- 2021-11-01 05:50
- HOME Server will be put offline to start a repair of a filesystem inconsistence
Linux printing affected by PrintNightmare vulnerability patch
Status:
- 2021-07-05 13:00
- ID resolved the issue
- 2021-07-05 09:41
Workaround: Use platform-independent printing
- 2021-07-05 09:41
- Authentification for printing fails. Ticket at ID servicedesk opened.
Downtime various D-ITET services for server maintenance
Status:
- 2021-04-27 08:30
- Condor is back online, all services restored.
- 2021-04-27 08:15
- Matrix/Element Chat services back online.
- 2021-04-27 08:00
- Database upgrade done and online.
- 2021-04-27 07:30
- Slurm services are back online.
- 2021-04-27 07:00
- Base system has been upgraded, main database services in progress of upgrade.
- 2021-04-27 06:00
- On 2021-04-27 between 06:00 and 08:30 ISG is going to update a server providing access to various D-ITET services. During the migration the following services will be affected and offline:
- Matrix/Element Chat services (the instances will be unavailable)
- IFA/Control Website: Access to the IFA database is blocked
- Slurm (D-ITET Arton Cluster): It won't be possible to submit new jobs or view Slurm statistics. Already running jobs will not be affected.
- Condor: Condor clients will be shut down the evening before to avoid running jobs during the migration.
Network disruption affecting several ISG.EE services
Status:
- 2021-03-31 09:30
The configuration error was found. The configuration change will be deployed on 2021-04-01 around 06:15 and a short network of about 1min is expected.
- 2021-03-31 08:00
- ID Networking team has rolled-back a deployed configuration, pending further investigation/analysis.
- 2021-03-31 07:30
There are currently disruption affecting a VPZ with servers managed by ISG.EE. Networking team of ID is investigating the issue. There are several ISG.EE services affected/malfunctioning due to this in particluar the FindYourData service.
login.ee.ethz.ch: downtime for server upgrade
Status:
- 2021-03-11 06:30
- Upgrade completed and service is up and running again.
- 2021-03-11 06:00
- The server servicing login.ee.ethz.ch will be upgraded to a new OS version (Debian buster). During the time of the update logins might not be possible.
Planned project/ archive storage downtime and client reboot
Status:
- 2020-07-11 12:00
- Migration has been completed, all services are back to operational state.
- 2020-07-11 08:00
- Migration started, services are shutdown
- 2020-07-11 8:00-12:00
- Start of planned maintenance work. Project/ archive storage services (known under the names "ocean", "bluebay", "lagoon" and "benderstor") will not be available. ISG-managed Linux clients will be rebooted.
svn.ee.ethz.ch downtime for server upgrade
Status:
- 2020-06-04 07:05
- Webservices for managing SVN repositories are enabled.
- 2020-06-04 06:15
Systemupgrade is done and access to the SVN repositories via the svn and https transport protocols are back online.
- 2020-06-04 06:00
- The server servicing the SVN repositories will be upgraded to a new operating system version. During this timeframe outages for access to the SVN repositories are expected.
European HPC cluster abuse
Status:
Recently European HPC clusters have been attacked and abused for mining purposes. The D-ITET Slurm and SGE clusters have not been compromised. We are monitoring the situation closely.
- 2020-05 17 08:30
- No successful login from known attacker IP addresses could be determined, none of the files indicating being compromised have been found on our file systems
- 2020-05-16 14:30
- No unusal cluster job activity was observed
D-ITET Netscratch downtime for server upgrade
Status:
- 2020-05-04 06:00
- Server upgrade has been completed.
- 2020-05-04 06:00
- The server servicing the D-ITET Netscratch service will be upgraded to a new operating system version. During this timeframe outages for the NFS service will be expected.
Network outage ETx router
Status:
- 2020-04-07 05:30
There was an issue on the Router rou-etx. ID networking team trackled and solved the issue. There was about a 10min interuption for the ETx networking zone affecting almost all ISG.EE maintained systems.
login.ee.ethz.ch: Reboot for maintenance
Status:
- 2020-04-06 05:35
System behind login.ee.ethz.ch has been rebootet for maintenance and increase available resources.
See the information on access D-ITET resources remotely. To distribute better the load user are encouraged to use the VPN service whenever possible.
itet-stor (FindYourData) Server maintenance: Reconfiguration of VM parameters
Status:
- 2020-02-18 19:03
- System again up and running.
- 2020-02-18 19:00
Scheduled downtime for the itet-stor/FindYourData service due to maintenance work on the underlying server.
itet-stor (FindYourData) Server migration: New operating system version
Status:
- 2020-01-20 07:15
OS upgrade done, there were short interruptions to the itet-stor/FindYourData service.
- 2020-01-20 06:00
We will update the server servicing the FindYourData service from Debian jessie 8 to Debian stretch 9. There will be short downtimes accessing this service during the update.
Archived status reports
2010 2011 2012 2013 2014 2015 2016 2017 2018 2019