We are in the process of migrating this forum. A new space will be available soon. We are sorry for the inconvenience.

Incident:


PS4ALL
29-08-13, 21:14
Servers in RBX (RBX-1) zijn op dit moment wel bereikbaar, Fail-Over IP's zijn echter in de meeste gevallen niet bereikbaar.
------
For internal connection between the DCs, we have set up an internal dedicated network, which is isolated from the backbone. It allows us to communicate between client services within the DCs and inter-DC. It also enables the mitigation to make the 3 VACs work simultaneously, regardless of where the DST IP is hosted.

With the arrival of vRack 1.5, we have set up a new network between these same DCs, supported by vRack 1.5. In addition, this new internal network will take over from the previous internal network, which means communicating between servers, pCC, etc. regardless of the room, DC and inter-DC. And of course, the routing of packets already cleaned by the VACs.

We thus have to reconfigure the old network to the new network, and then shut down the old one.

Comment by OVH - Thursday, 29 August 2013, 10:19AM
We have also reconfigured the DC network in SBG.


Comment by OVH - Thursday, 29 August 2013, 10:21AM
Mitigation has moved onto to the new internal network.

The old SBG/RBX network has been shut down. We are switiching the links onto the new network. We will thus have full security between the DCs:
SBG/RBX by Frankfurt
SBG/RBX by Paris.

Comment by OVH - Thursday, 29 August 2013, 10:24AM
The old RBX/GRA network has been shut down. We are switiching the links onto the new network. We will thus have full security between the DCs:
RBX/GRA through Paris (to be upgraded to "through Bois Grenier")
RBX/GRA through London (to be upgraded to "through Brussels")

As part of above we are going to change the OSPF AREA of all the small routers in RBX1.

Comment by OVH - Thursday, 29 August 2013, 6:38PM
rbx-3/4/5/6-m1/m2 done


Comment by OVH - Thursday, 29 August 2013, 6:40PM
rbx-7/8/9/10/11/12/14/15/16/17-m1/m2: done there is some card damage on rbx-14, rbx-4, rbx-3. we are replacing with spare

Comment by OVH - Thursday, August 29, 2013, 8:13 p.m.
we ended routers vlan 1 (70 routers) but there is nothing wrong.

OSPF process not want to go back. on is trying to simplify the configuration annocer to avoid all LSA then restart routers that have crashed. was already some UP routers. but much remains to be done.


Comment by OVH - Thursday, August 29, 2013, 9:38 p.m.
while we try to trace the OSPF router by router is added to the BGP configuration OSPF turn these small routers.

Comment by OVH - Thursday, August 29, 2013, 11:00 p.m.
we still have two routers back.

Comment by OVH - Thursday, August 29, 2013, 11:07 p.m.
UP is on at least one router 2. OSPF is cut. and all but operates again on the BGP.

Comment by OVH - Thursday, August 29, 2013, 11:48 p.m.
We have the problem of the OSPF between all routers RBX. Obviously the problem of OSPF RBX1 has spread its problem on all other routeus. It is looking at whether a simple OSPF process restart can fix this worries that an operation must be heavier.

Comment by OVH - Friday, August 30, 2013, 0:09
The internal network is raised.