Netbox: Front/Rear port mapping breaks when multiple cables are between devices

[toddam310]

Environment

• Python version:
• NetBox version: 2.5.10

**

duplicate key value violates unique constraint "dcim_interface__connected_circuittermination_id_key"
DETAIL: Key (_connected_circuittermination_id)=(22) already exists.**

When I have two devices that I am mapping multiple front ports to a rear port on each and I join the rear ports of the two devices together, I can put a device on front port 1 on device A and it will connect all the way through to a device placed on front port 1 on device B. Same for any device I connect to a front port on Device A, it will show as connected to the corresponding port on device B.

F 1 ---\                        /---F 1
 .  ----\                      /---- .
 .  -----\____________________/----- .
 .  -----/rear 1:1    rear 1:1\----- .
 .  ----/     .          .     \---- .
F 10---/      .          .      \---F 10  
          rear 1:10   rear 1:10

Now, when I put any other cables via panels that are mapped as one front port to one back port (like a standard patch panel would be) between the rear ports of device A & B, I can again put a device on front port 1 on device A and it will connect all the way through to a device placed on front port 1 on device B. However, when I add a second device via a cable to the next front port on device A, I get the above error. Also, the device port will show up as cabled, even after the error, but the trace will not go to the intended device on the other end and will instead go to the first device built.

F 1  ---\                                            /---F 1
   . ----\                                          /---- .
   . -----\_________o_________o__________o_________/----- .
   . -----/rear 1:1  cable 101  cable 102  rear 1:1\----- .
   . ----/    .                               .     \---- .
F 10 ---/     .                               .      \---F 10  
           rear 1:10                       rear 1:10 

Steps to Reproduce

1. Create two devices of the same device type with multiple front ports mapping to a single back port.
2. Join the two device's rear ports together via panels that simply map one front port to one back port.
3. Start putting cables between the front ports on both devices. The first one should trace through, any subsequent ones will generate and error, appear to have been added anyway, but will not trace through properly.

Expected Behavior

I was expecting a something connected on device A port 2, to be able to connect to Device B port 2 regardless of whether there are additional cables in between.

Observed Behavior

Instead I get an error when adding a second connection (first connection will work) and the resulting cable's trace will not map correctly.

[DanSheps]

Does the middle device/panel have 10 front/10 rear ports as well?

The integrity error points to you trying to create a cable where a circuit exists.

[toddam310]

No, the panels can be anywhere from 24 to 48 ports (front and back) and are one-to-one mapped front to rear. There are no circuits in between.

[DanSheps]

I am unable to reproduce this. Please confirm it is modeled correctly here:

https://master.netbox.dansheps.com/dcim/devices/5/

[fablabo]

Does the middle device/panel have 10 front/10 rear ports as well?

The middle device connecting cable 101 and cable 102 probably only has one front and one rear port. This results in a single path. I've created PP1, PP2 and PP3 on your demo install to reproduce the error.
Creating cable number 30 did throw an exception and a trace also fails.

To allow multiple connections between two panels which have a one to many port relation, a multi position front port could be a solution.

A real life situation for this setup would be a cable plant which uses MPO patches.

[DanSheps]

I don't think you would generally go rear port to front port. It owuld normally be front:rear-rear:front-front:rear-rear:front

[fablabo]

MPO patching is becoming quite common where you heave a breakout (front:rear 12:1 or 24:1) on the far and near end, and N+1 MPO patchpanels in between. For this you'll need 12:12 or 24:24 front:rear ports.

[jeremystretch]

@fablabo Please expand on the steps to reproduce, noting the exact objects to create/update in each step. This will allow us to try recreating the reported issue.

[fablabo]

Add Site

• Name: AcmeSite
• Status: Active

Add Manufacturers

• Name: Acme

Add Device Role

• Name: Patchpanel

Create MPO to 12x LC breakout box

Add Device type

• Manufacturer: Acme
• Model: MPOBreakout12LC
• Height: 1U
• Full Depth: No

Add Rear Port

• Device: MPOBreakout12LC
• Name: MPO1
• Type: MPO
• Positions: 12

Add Front Port

• Device: MPOBreakout12LC
• Name: LC[1-12]
• Type: LC
• Reaports: MPO1:[1-12]

Create MPO Patchpanel

Add Device type

• Manufacturer: Acme
• Model: MPOPatchfarm
• Height: 1U
• Full Depth: No

Add Rear Port

• Device: MPOPatchfarm
• Name: MPOrear[1-4]
• Type: MPO
• Positions: 1

Add Front Port

• Device: MPOPatchfarm
• Name: MPOfront[1-4]
• Type: MPO
• Rear ports: MPOrear[1-4]:1

Create generic switch

Add Device Type

• Manufacturer: Acme
• Model: Switch
• Height: 1U
• Is full depth: Yes

Add Interfaces

• Device: Switch
• Name: [1-24]
• Form Factor: SPF+

Create patchpanels

Add Device

• Name: PP1
• Device role: Patchpanel
• Manufacturer: Acme
• Device Type: MPOBreakout12LC
• Site: AcmeSite

Add Device

• Name: PP2
• Device role: Patchpanel
• Manufacturer: Acme
• Device Type: MPOBreakout12LC
• Site: AcmeSite

Add Device

• Name: Patchfarm1
• Device role: Patchpanel
• Manufacturer: Acme
• Device Type: MPOPatchfarm
• Site: AcmeSite

Add Device

• Name: Switch1
• Device Role: Access Switch
• Manufacturer: Acme
• Device Type: Switch
• Site: AcmeSite

Add Device

• Name: Switch1
• Device Role: Access Switch
• Manufacturer: Acme
• Device Type: Switch
• Site: AcmeSite

Connect:

• PP1 (rear) MPO1 to Patchfarm1, Rear port, MPOrear1
• PP2 (rear) MPO1 to Patchfarm1, Rear port, MPOrear2
• Patchfarm1 MPOFront1 to Patchfarm1, Front port, MPOfront2
• Switch1 interface 1 to PP1, front port LC1
• Switch2 interface 1 to PP2, front port LC1

At this stage Switch 1 Interface 1 connects to Switch 2 Interface 1. A trace shows 5 cables, both ending on a SFP+ port.

Now we add a second cable from Switch 2 Interface 2 to PP2 LC2 which results in an error:

Internal Server Error: /dcim/interfaces/764/connect/

Exception at /dcim/interfaces/764/connect/
Invalid position for MPOrear2 (1 positions): 2)

The new cable is added but viewing or deleting it results in a similar error.
Ending up in a loop where you can't add or edit a cable is unfortunate but understandable. MPOrear2 only has 1 position while MPO1 on PP2 has 12.

To continue we can increase the amount of positions on both MPOrear ports on Patchfarm1.

Next we add a connection between Switch1, Interface 2 to PP1, front port LC2 just like we did between Switch2 and PP2 front port LC2.

This results in the following error:

Internal Server Error: /dcim/interfaces/740/connect/

ValueError at /dcim/interfaces/740/connect/
Connected endpoint must be an Interface or CircuitTermination, not <class 'dcim.models.RearPort'>.

Again the cable is added but the trace between Switch1 interface 2 and Switch 2 interface 2 only shows tw0 cables up-to the MPOrear port.

Next we can delete all cables (without errors) and the same cables can be recreated without errors when they are added in a different order.

Connect:

• Switch1 Interface 1 to PP1 front port LC1
• Switch1 Interface 2 to PP1 front port LC2
• Switch2 Interface 1 to PP2 front port LC1
• Switch2 Interface 2 to PP2 front port LC2
• PP1 Rear port MPO1 to Patchfarm1 rear port MPOrear1
• PP2 Rear port MPO1 to Patchfarm1 rear port MPOrear2
• Patchfarm 1 MPOfront1 to Patchfarm front port MPOfront2

This results in the same but without errors. Interface 2 on both switches do show a cable but the Connection column shows Not connected.

As I've already mentioned before the result is very understandable. In both procedures the first connection made connects fully from near to far. In the first procedure a connection ends on the 'leaving' MPO1 port which has multiple position but the connecting MPOrear[1-2] only has one position. In the second procedure a connection ends on the MPOrear[1-2] port which has multiple position but only one of them is connected to the front port.

Without any knowledge on the internals of netbox I'm guessing that a solution could either be a fully transparent mapping between front and rear ports for a certain type (no positions), or a front port with multiple positions.

The error handling could be improved by not allowing a second connection on a path which doesn't have sufficient "positions", but the result doesn't seem to be a bug/issue, just the way ports and cables are currently setup.

Providing a way to model a transparent front/rear paths or a front port with multiple positions would be a great feature to allow for WDM devices and MPO patchfarms. But that would make this a feature request, not an issue.

[steffann]

I got the same problem when modelling two CWDM/DWDM muxes connected over a circuit:

F 1  ---\                                 /---F 1
   . ----\                               /---- .
   . -----\_________A_________Z_________/----- .
   . -----/rear 1:1  circuit 1  rear 1:1\----- .
   . ----/    .                    .     \---- .
F 10 ---/     .                    .      \---F 10  
           rear 1:10            rear 1:10 

When connecting the first device interface to the left mux (say to F1) everything is fine. It stores the circuit termination ID of the A side of the circuit in _connected_circuittermination_id. However when connecting the second device interface to the same mux (say to F2) the duplicate key value violates unique constraint "dcim_interface__connected_circuittermination_id_key" DETAIL: Key (_connected_circuittermination_id)=(22) already exists error appears as it finds the same circuit termination ID and tries to store it in _connected_circuittermination_id. Because it is a OneToOneField that is not allowed.

I don't see any issue with changing _connected_circuittermination_id to ForeignKey though, which solves the problem. I tested this in my own setup and it works fine. I'll submit a pull request for this change.

[fablabo]

@steffann Have you tried the trace feature after your change and adding multiple interfaces to the muxes ?

For the trace feature to work it would have to "remember" the trace left e.g. port F3 on the left, trace the circuit, enter the next rear port and then exit F3 on the right.

[steffann]

It actually works great! I didn't expect it, but I tested a quite complex setup:

Device 1 Te3/4 - [F1 -\      ]               [      /- F1] - [F1 -\      ]               [      /- F1] - Device 2 Te6/2
                 [     - Rear] - Circuit A - [Rear -     ]   [     - Rear] - Circuit B - [Rear -     ]
Device 1 Te3/6 - [F2 -/      ]               [      \- F2] - [F2 -/      ]               [      \- F2] - Device 2 Te6/3

So two connections from device 1 to device 2 over two circuits, each with a DWDM Mux on both sides. The traces work fine! Screenshots attached.

[fablabo]

@steffann Your fix doesn't seem to do the trick for me. When I try to re-create your setup i'm seeing the same connection in the Interfaces table of the Device view. So on Device 1 both Te3/4 and Te3/6 show a connection to Device 2 Te6/2 and Device 2 shows both Te6/2 and Te6/3 connected to Te3/4.

The trace does work and shows all interfaces correctly.

When creating the following setup using front/rear connections instead of a Circuit, all traces end up on the first port on the far end.

F 1  ---\                                            /---F 1
   . ----\                                          /---- .
   . -----\_________o_________o__________o_________/----- .
   . -----/rear 1:1  cable 101  cable 102  rear 1:1\----- .
   . ----/    .                               .     \---- .
F 10 ---/     .                               .      \---F 10  
           rear 1:10                       rear 1:10 

[steffann]

Hi,

@steffann Your fix doesn't seem to do the trick for me. When I try to re-create your setup i'm seeing the same connection in the Interfaces table of the Device view. So on Device 1 both Te3/4 and Te3/6 show a connection to Device 2 Te6/2 and Device 2 shows both Te6/2 and Te6/3 connected to Te3/4.

The trace does work and shows all interfaces correctly.

Ah right, I hadn't looked at the cached interface info. I'll check where that information is filled in and fix that.

Cheers,
Sander

[steffann]

Hi,

Op 16 jun. 2019, om 20:53 heeft Sander Steffann [e-mail] het volgende geschreven:

Hi,

@steffann Your fix doesn't seem to do the trick for me. When I try to re-create your setup i'm seeing the same connection in the Interfaces table of the Device view. So on Device 1 both Te3/4 and Te3/6 show a connection to Device 2 Te6/2 and Device 2 shows both Te6/2 and Te6/3 connected to Te3/4.

The trace does work and shows all interfaces correctly.

Ah right, I hadn't looked at the cached interface info. I'll check where that information is filled in and fix that.

And done. Can you please try again with the latest patch?

Cheers!
Sander

[fablabo]

Thanks for the effort @steffann but unfortunately no joy so far. On the near end I've connected 9 Gi ports to 9 different cwdm channels. on the fear end only the first two channels are connected.

[steffann]

Hi,

Thanks for the effort @steffann but unfortunately no joy so far. On the near end I've connected 9 Gi ports to 9 different cwdm channels. on the fear end only the first two channels are connected.

Did you re-create the cables? The change is when saving the cable. Doing this might help:

python3 manage.py shell

from dcim.models import *
for cable in Cable.objects.all():
... cable.save()

Cheers,
Sander

[fablabo]

I did a clean install with #3254

Fast-forward
 netbox/dcim/migrations/0070_auto_20190612_1801.py | 24 ++++++++++++++++++++++++
 netbox/dcim/migrations/0071_auto_20190612_1730.py | 19 +++++++++++++++++++
 netbox/dcim/models.py                             |  5 ++---
 netbox/extras/models.py                           | 25 +++++++++----------------
 4 files changed, 54 insertions(+), 19 deletions(-)
 create mode 100644 netbox/dcim/migrations/0070_auto_20190612_1801.py
 create mode 100644 netbox/dcim/migrations/0071_auto_20190612_1730.py
root@netbox-dev:/opt/netbox# python3 netbox/manage.py makemigrations --merge
Merging dcim
  Branch 0070_auto_20190617_1242
    - Alter field termination_a_type on cable
    - Alter field termination_b_type on cable
  Branch 0071_auto_20190612_1730
    - Alter field _connected_interface on interface
    - Alter field _connected_circuittermination on interface
    - Alter field _connected_circuittermination on interface

Merging will only work if the operations printed above do not conflict
with each other (working on different fields or models)
Do you want to merge these migration branches? [y/N] Y

Created new merge migration /opt/netbox/netbox/dcim/migrations/0072_merge_20190617_1245.py
root@netbox-dev:/opt/netbox# python3 netbox/manage.py migrate
Operations to perform:
  Apply all migrations: admin, auth, circuits, contenttypes, dcim, extras, ipam, secrets, sessions, taggit, tenancy, users, virtualization
Running migrations:
  Applying dcim.0070_auto_20190612_1801... OK
  Applying dcim.0071_auto_20190612_1730... OK
  Applying dcim.0070_auto_20190617_1242... OK
  Applying dcim.0072_merge_20190617_1245... OK

[steffann]

Hi Fabian,

I did a clean install with #3254

Sorry, apparently I forgot to push my branch facepalm

Please try again
Sander

[fablabo]

Haha no worries. It's starting to look very promising!

SwitchA

SwitchB

Trace

A minor cosmetic issue in the termination of the Circuit

I'm thinking this should be the common ports on both muxes.

3254 is a great change to allow for a Circuit between multiposition rear-ports, but it isn't yet a fix for the issue described by @toddam310 where multiple cables and front/rear ports are used between multiposition rear-ports.

[steffann]

Ok, my patch got rejected. Let's continue the discussion on what the proper solution is.

[steffann]

I suggest using a subset of what I proposed as a patch:

• Change dcim.Interface._connected_circuittermination to a ForeignKey instead of OneToOne
• Change dcim.cable.get_path_endpoints to follow circuits

That way there won't be a change to the endpoint data model, CWDM and DWDM circuits will work, and endpoints will show the correct peer endpoint.

If this is an acceptable solution I can have a clean patch set ready in minutes. The development and alpha testing has been done already in my previous patch proposal :)

[steffann]

This time I'll wait with clicking the pull request button until we have agreement :)

[steffann]

I just realised what @fablabo has been trying to tell me: these are two similar but slightly different problems, and my solution doesn't solve the original question. I will therefore move my stuff to a separate issue.

[steffann]

FYI: I can fix this as part of #3633.

[steffann]

The case where a rear port has only one position (basically extending the cable) raises an exception. This is caused by such 1-on-1 ports being pushed onto the position_stack, even though there doesn't have to be a corresponding rear port to pop it again.

[jeremystretch]

@steffann please open a new issue for this.