Support for other switches

[CJBuchel]

The Cisco lineup works well enough for the application, but the old 3560G is long out of service. This along with it needing to be purchased second hand makes it a bit of a shot in the dark, at least in my area, I can never find 2 of the same models for redundancy or ease of mind. (Often times these switches come out of datacenters and may have been pushed fairly hard in their lifetime). And buying new cisco switches is not very cost effective or valuable for what you get.

Other similar models work fairly well with some tweaking (i.e, we use a set of 3750G's) but requires a fair amount of reconfiguring. And on rare occasions, modifications to the Go code. In our case, it was just a tweak to the config script. But due to Cisco shenanigans, it also runs on older firmware, with no easy way of updating the firmware to something later, i.e v15.0 used in the cheesy configuration.

Recently I've thought more and more about trying to get different switches to work, like TP-Link and Ubiquity which can be bought new for a reasonable price with modern hardware and software, fairly easily upgradible and in the case of Unify, also managable through a controller. In our case Unify products seem to work well for our needs, as we have a few Unify AP's for team internet and admin/FMS networking. And a dream machine for the gateway, also acting as the unify controller for site management. Which makes quick venue setups fairly easy to configure all the networking at the start of the day, without needing to individually touch the switches or ap's scattered around the venue. Especially for changes during the event.

In the coming months I was going to play around and see if I could integrate the Unify API into cheesy
https://ubntwiki.com/products/software/unifi-controller/api
Unify switches also have the same SSH style as cisco, but it seems more flexible to use the top layer API from the controller, and with the Radio now using an API, having the switch also using an API just seems nice to abstract away the nitty gritty.

It got me thinking that it would be nice to see officially in Cheesy as well, perhaps a dropdown for the type of switch/networking in the setup page to allow for different support.

Curious on the thoughts about it, I'll be trying it out in my own spare time for Unify. But I thought I'd chuck it down as a Feature request/Issue.

[justfishin1999]

I agree... would be nice to have something that's not 15+ years old and is still in support and readily available.

With that said - I've done some work on this recently, and was able to get an alternate switch working. I'm currently looking at building this out for HP and Aruba switches. I built a selector into Cheesy Arena's settings that you can choose which Vendor/Model of device that you wish to use, and it will then run alternate functions to configure the respective switch. I'm still doing some testing with this, but hope to have it robust enough to share.

The tough part with Unifi is that you'd have to have a UDM or something running a controller alongside of the switch. I'm not sure how well the API would work with the controller, if there's any type of latency or delay from FMS > Controller > Switch could make things a little interesting. I am personally not sure I want to put the time into TP-Link switches because I'm not totally convinced they would have adequate performance, plus their CLI is not that great either.

[CJBuchel]

Ah, funny you should mention it.
I did recently get the Ubiquiti system working within my own tests, it hasn't been fully cleaned up yet but the configuration works quite well. I haven't tried to do any multi selector vendor in case the ubiquiti system broke and I needed a backup (currently hard coded for now).

But the Ubiquiti method works surprisingly well, it's all API based and I've noticed a significant speed increase when configuring the switches. But it does have both the good and bad requirements of a unifi console/unifi virtual controller. In our case it was fine because we have an offseason box, which includes a few ubiquiti switches and a UDM Pro as the gateway router for internet connections.

Overall it allows the configuration to be fairly flexible, as the FMS only ever needs to communicate to the main unifi controller never any switches directly, whether it be running in a virtual container, an rpi or a proper UDM. And all changes propogate to the rest of the unifi network. In theory you could setup multiple unifi switches all over the field with tagged networking ports. Such as 10,20,30 etc... And then all changes from the FMS would propagate to them just as easily. It offloads the specific configuration from the FMS code to instead the unifi controller, making it easier to completely rebuild the network depending on the event. And all the switches (if tagged correctly) would still function as expected. For instance a single score switch taking in all team driverstations, or dual switches for red and blue alliances.

The only issue is that it is more complicated to setup initially, especially because it needs some kind of unifi controller. Virtual or physical. Which may drive up costs for some. I'll share a snippet of the code I changed up for the unifi system. It's not fancy but it gets the job done in our scrimmage tests. And instead of calling the standard cisco config, I instead call this one.

package network

import (
	"bytes"
	"crypto/tls"
	"encoding/json"
	"fmt"
	"io"
	"net/http"
	"net/http/cookiejar"
	"strings"
	"sync"
	"time"

	"github.com/Team254/cheesy-arena/model"
)

// const (
// 	switchConfigBackoffDurationSec = 1
// 	switchConfigPauseDurationSec = 1
// 	switchTeamGatewayAddress       = 4
// )

type UnifiNetwork struct {
	address string
	password string
	csrfToken string
	mutex sync.Mutex
	configBackoffDuration time.Duration
	configPauseDuration time.Duration
	Status string
}

var UdmIpAddress = "10.0.100.1" // The UDM is the gateway router
var authRoute = "api/auth"
var defaultRoute = "proxy/network/api/s/default"
var restRoute = fmt.Sprintf("%s/rest", defaultRoute)
var statRoute = fmt.Sprintf("%s/stat", defaultRoute)

func NewUnifiNetwork(address, password string) *UnifiNetwork {
	return &UnifiNetwork{
		address:                 address,
		password:                password,
		configBackoffDuration:   1 * time.Second,
		configPauseDuration:     1 * time.Second,
		Status:                  "UNKNOWN",
	}
}

// Login for the UDM Router
func (ubnt *UnifiNetwork) login(client *http.Client) error {
	// Create login request
	loginData := map[string]string{
		"username": "root", // Replace with your username
		"password": ubnt.password, // Replace with your password
	}
	jsonData, _ := json.Marshal(loginData)
	
	req, _ := http.NewRequest("POST", 
		fmt.Sprintf("https://%s/%s/login", ubnt.address, authRoute), 
		bytes.NewReader(jsonData))
	req.Header.Set("Content-Type", "application/json")
	
	// Send login request
	resp, err := client.Do(req)
	if err != nil {
		return err
	}
	defer resp.Body.Close()
	
	// Check for success
	if resp.StatusCode != http.StatusOK {
		body, _ := io.ReadAll(resp.Body)
		return fmt.Errorf("login failed with status %d: %s", resp.StatusCode, string(body))
	}
	
	// Get CSRF token from response header
	csrfToken := resp.Header.Get("X-CSRF-Token")
	// Set network token
	ubnt.csrfToken = csrfToken
	return nil
}

func (ubnt *UnifiNetwork) getNetworkID(client *http.Client, vlanID int) (string, error) {
	// Create request to get networks
	req, _ := http.NewRequest("GET", fmt.Sprintf("https://%s/%s/networkconf", ubnt.address, restRoute), nil)
	req.Header.Set("X-CSRF-Token", ubnt.csrfToken)
	
	// Send request
	resp, err := client.Do(req)
	if err != nil {
		fmt.Println("Error sending request:", err)
		return "", err
	}
	defer resp.Body.Close()
	
	// Check for success
	if resp.StatusCode != http.StatusOK {
		return "", fmt.Errorf("failed to get networks with status %d", resp.StatusCode)
	}

	// Read the response body for debugging
	bodyBytes, err := io.ReadAll(resp.Body)
	if err != nil {
		return "", fmt.Errorf("failed to read response body: %v", err)
	}
	
	// Parse response - first as a generic map to see the structure
	var responseObj map[string]interface{}
	if err := json.Unmarshal(bodyBytes, &responseObj); err != nil {
		return "", fmt.Errorf("error parsing response: %v", err)
	}

	// Find the network with the specified VLAN ID
	for _, network := range responseObj["data"].([]interface{}) {
		networkMap := network.(map[string]interface{})

		if networkMap["vlan"] != nil && networkMap["vlan"].(float64) == float64(vlanID) {
			return networkMap["_id"].(string), nil
		}
	}

	// If not found, return an error
	return "", fmt.Errorf("network with VLAN ID %d not found", vlanID)
}

type TeamNetworkConfig struct {
	networkID string
	enabled bool
	vlan int
	dhcpIP string // DHCP IP address e.g 10.47.88.1
}

func (ubnt *UnifiNetwork) updateTeamNetwork(client *http.Client, teamConfig TeamNetworkConfig) error {

	// split the dhcpIP into parts and replace last part with 50
	dhcpParts := strings.Split(teamConfig.dhcpIP, ".")
	if len(dhcpParts) != 4 {
		return fmt.Errorf("invalid DHCP IP address format: %s", teamConfig.dhcpIP)
	}
	gateway_address := strings.Join(dhcpParts[:3], ".") + ".4" // Set the last part to 4
	dhcp_start := strings.Join(dhcpParts[:3], ".") + ".50" // Set the last part to 50
	dhcp_stop := strings.Join(dhcpParts[:3], ".") + ".200" // Set the last part to 254

	// Network configuration data
	networkConfig := map[string]interface{}{
		"vlan_enabled":   true,
		"vlan":           teamConfig.vlan,
		"ip_subnet":      teamConfig.dhcpIP+"/24",
		"dhcpd_enabled":  true,
		"dhcpd_start":    dhcp_start,
		"dhcpd_stop":     dhcp_stop,
		"enabled":        teamConfig.enabled,
		"gateway":        gateway_address,
	}
	jsonData, _ := json.Marshal(networkConfig)

	// Create update request
	url := fmt.Sprintf("https://%s/%s/networkconf/%s", ubnt.address, restRoute, teamConfig.networkID)
	req, _ := http.NewRequest("PUT", url, bytes.NewReader(jsonData))
	req.Header.Set("Content-Type", "application/json")
	req.Header.Set("X-CSRF-Token", ubnt.csrfToken)

	// Send update request
	resp, err := client.Do(req)
	if err != nil {
		return err
	}
	defer resp.Body.Close()

	// Check for success
	if resp.StatusCode != http.StatusOK {
		body, _ := io.ReadAll(resp.Body)
		return fmt.Errorf("update failed with status %d: %s", resp.StatusCode, string(body))
	}

	return nil
}

func (ubnt *UnifiNetwork) getDeviceStatus(client *http.Client) (bool, error) {
	req, _ := http.NewRequest("GET", fmt.Sprintf("https://%s/%s/device", ubnt.address, statRoute), nil)
	req.Header.Set("Content-Type", "application/json")
	req.Header.Set("X-CSRF-Token", ubnt.csrfToken)

	// Sendrequest
	resp, err := client.Do(req)
	if err != nil {
		return false, fmt.Errorf("error getting device status: %w", err)
	}
	defer resp.Body.Close()

	// Check for success
	if resp.StatusCode != http.StatusOK {
		return false, fmt.Errorf("failed to get device status with status %d", resp.StatusCode)
	}

	// Parse response
	var result map[string]interface{}
	if err := json.NewDecoder(resp.Body).Decode(&result); err != nil {
		return false, fmt.Errorf("error parsing device status response: %w", err)
	}

	// Extract device statuses
	devices, ok := result["data"].([]interface{})

	if !ok {
		return false, fmt.Errorf("unexpected data format in device status")
	}

	for _, dev := range devices {
		device, ok := dev.(map[string]interface{})
		if !ok {
			fmt.Print("Skipping status check (unknown issue)")
			continue
		}

		// 0 = Disconnected/Offline
		// 1 = Connected/Online
		// 2 = Pending Adoption
		// 3 = Adopting
		// 4 = Adoption Failed/Heartbeat Missed
		// 5 = Provisioning ("Getting Ready")
		// 6 = Managed (Fully Ready/Configured)

		// print line by line
		if device["type"] == "udm" {
			state := int(device["state"].(float64))
			successful := device["last_config_applied_successfully"].(bool)
			if state == 1 && successful { // json decodes numbers as float64
				return true, nil
			}
		}
	}

	return false, nil
}

func (ubnt *UnifiNetwork) ConfigureTeamNetwork(teams [6]*model.Team) error {
	// Lock the mutex to prevent concurrent access
	ubnt.mutex.Lock()
	defer ubnt.mutex.Unlock()
	ubnt.Status = "CONFIGURING"

	// Create HTTP client with cookie jar
	jar, err := cookiejar.New(nil)
	if err != nil {
		fmt.Println("Error creating cookie jar:", err)
		ubnt.Status = "ERROR"
	}

	client := &http.Client{
		Jar: jar,
		Transport: &http.Transport{
			TLSClientConfig: &tls.Config{InsecureSkipVerify: true},
		},
	}

	// Log into the UDM router
	if err := ubnt.login(client); err != nil {
		fmt.Println("Error logging in:", err)
		ubnt.Status = "ERROR"
		return err
	}

	// Set team networks to disabled with default DHCP IPs
	for vlan := 10; vlan <= 60; vlan += 10 {
		networkID, err := ubnt.getNetworkID(client, vlan)
		if err != nil {
			fmt.Println("Error getting network ID:", err)
			ubnt.Status = "ERROR"
			return err
		}

		teamConfig := TeamNetworkConfig{
			networkID: networkID,
			enabled:   false,
			vlan:      vlan,
			dhcpIP:    fmt.Sprintf("10.0.%d.4", vlan),
		}
		if err := ubnt.updateTeamNetwork(client, teamConfig); err != nil {
			fmt.Println("Error updating team network:", err)
			ubnt.Status = "ERROR"
			return err
		}
	}

	// Set the team networks
	setTeamNetwork := func(team *model.Team, vlan int) error {
		if team == nil {
			return nil
		}
		teamPartialIp := fmt.Sprintf("%d.%d", team.Id/100, team.Id%100)
		dhcpIP := fmt.Sprintf("10.%s.4", teamPartialIp)
		networkID, err := ubnt.getNetworkID(client, vlan)
		if err != nil {
			fmt.Println("Error getting network ID:", err)
			ubnt.Status = "ERROR"
			return err
		}
		teamConfig := TeamNetworkConfig{
			networkID: networkID,
			enabled:   true,
			vlan:      vlan,
			dhcpIP:    dhcpIP,
		}
		if err := ubnt.updateTeamNetwork(client, teamConfig); err != nil {
			fmt.Println("Error updating team network:", err)
			ubnt.Status = "ERROR"
			return err
		}

		// print the status of the update for team
		fmt.Printf("Team %d VLAN %d Gateway: %s\n", team.Id, vlan, dhcpIP)
		return nil
	}

	for i, team := range teams {
		vlan := 10 * (i + 1)
		if team == nil {
			continue
		}
		fmt.Printf("Configuring team %d with VLAN %d\n", team.Id, 10*(i+1))
		if err := setTeamNetwork(team, vlan); err != nil {
			fmt.Println("Error setting team network:", err)
			ubnt.Status = "ERROR"
			return err
		}
	}

	// // wait for unifi to process the changes
	time.Sleep(time.Second * 1)

	// timeout
	timeout := time.Now().Add(30 * time.Second)
	
	// loop and check device status
	for time.Now().Before(timeout) {
		online, err := ubnt.getDeviceStatus(client)
		if err != nil {
			fmt.Println("Error getting device status:", err)
			ubnt.Status = "ERROR"
			return err
		}
		if online {
			fmt.Println("Network config complete")
			ubnt.Status = "ACTIVE"
			return nil
		} else {
			ubnt.Status = "CONFIGURING"
		}
		time.Sleep(time.Second)
	}

	// If we reach here, the device is not online
	fmt.Println("Network config timeout")
	ubnt.Status = "ERROR"
	return fmt.Errorf("network config timeout")
}

But something else I thought about recently is how the official FMS configures networks.
https://fms-manual.readthedocs.io/en/latest/fms-whitepaper/fms-whitepaper.html

Unlike in cheesy where the scoreswitch both has tagged ports and is reconfigured for those ports to provide a DHCP server that hands out IP addresses for each VLAN/team. The actual FMS as far as I can tell only has the ports tagged, similar to the SCC switches. And I believe the FMS server itself hosts the DHCP servers for all 6 teams.

To me this sounds like a very clean and flexible solution. As it would likely be compatible with any L3 switch that could tag the ports without any cheesy code change. And depending on how it was implemented, could offload the config as a proper code configuration by the FMS or API. Likely making it quite a bit quicker than the normal Cisco switch updates. And probably would provide more information back for errors if any.

It's a neat idea but I've not looked at it much. I'm also unsure on the best approach to do it, I've briefly looked into it and in some cases it seems like it could be built into the cheesy code using a DHCP server package or a custom written one. But I've also seen In other cases people have run a small rpi or linux computer hosting a DHCP server, and on the same machine serve a small API to access and configure that DHCP server to work.

Although our setup is currently done in a proxmox server where the FMS is an LXC container. And I'm unsure how well this method would work virtualised as another container. Still, it's a cool idea and one that I'll definitely try to look into eventually. But for now, unifi is our current setup for our August event, but maybe afterwards, I can have a crack and share my findings.

[FletcherS7]

Managing a DHCP server on the operating system would vastly increase the complexly of the Cheesy Arena installation and usage, Running a DHCP server on the operating system is a lot more overhead then you think.

Also keep in mind that the Score Switch (both with Cheesy Arena and FIRST FMS) also has access list's that control what traffic is allowed to leave the team VLAN's and does all the VLAN routing, so you will still need a proper layer 3 switch, not just a managed switch.

[CJBuchel]

True, I don't expect it to be an easy setup nor very feasible built into cheesy. And if anything would probably be a separate program running on a pi or pc which cheesy would access via an API. But even then it's an extra component which would be needed for an FMS setup.

My thinking is that while it doesn't eliminate the layer 3 switch requirements, it does mitigate code changes needed when you use a different switch. Most events will need to configure a switch prior to the event anyway, including setting up the tagged ports, the ACL's and any other special needs they might have. But the pre-match configuration sent by the FMS would no longer be locked to a Cisco switch. So an event could have options as to what switch they wanted to use, and it would likely work without any code changes to Cheesy. As even some Cisco switches other than the 3560G don't fully work without code changes to the underlying commands sent from Cheesy.

It's a bit of niche issue, but it's food for thought. As I think adding an extra bit of code for every brand of switch in a dropdown, while greatly beneficial. Likely would also get complex depending on the amount of switches you would try to support.

In an ideal world for me, I'd have a dropdown for 3 options. Cisco (default), Unify and then DHCP API.

[FletcherS7]

As even some Cisco switches other than the 3560G don't fully work without code changes to the underlying commands sent from Cheesy.

Any Cisco switch running IOS 12 or newer (with a IP-Base or higher license) will work with the commands sent from Cheesy Arena, that should cover any Cisco layer 3 switch made after 2005.
If you are trying to use something made before 2005 I suggest you reconsider.

[CJBuchel]

should cover any Cisco layer 3 switch made after 2005.

Yep, I don't disagree but I don't buy Cisco switches first hand and I doubt anyone else will either.
In my case I bought a set of 3750G's, the series release date was 2003, but the 3560G is 2004. So my assumption like any other, is that it was using something newer than IOS 12. Luckily it was, but twice before I've run into trying to buy a Cisco switch for cheesy and falling into the trap of it needing a different config. And then a mild code change depending on the OS version just to get it to work near enough. And sadly (at least for me) there isn't a decent way to tell until you have it on hand and start testing.

The switch that cheesy references is the 3560G. It's from 2004, it's not out there to assume someone will try and get the same or near to the same model for compatibility sake. I tried at least, and I fell short twice before getting the 3750G's. Which have their own problems, as they're stack switches and need a bit of a tweak to the configuration. But luckily no code changes.

I'm not arguing with the current solution, I just think it's old. And I've run into switch problems more often than not to be considered just a me issue. And the solution I don't think should be "buy more until you get one that works".

I'd rather just buy correctly the first time round. Which is why I'd like to see more compatibility with other brands, or something newer built into cheesy.

[FletcherS7]

Correction.
Any switch made after 1999.
I misremembered when IOS 12 released.

I would be curious what code changes you had to make in Cheesy Arena.
Changes to the IOS startup config is expected since interface numbering is very switch specific., but you should not need to modify the commands in Cheesy Arena.

[CJBuchel]

Hm, hang on. I'll try see if I have the old configs around.
It was a few years ago with a standard 3560G model.
And then an 8 port 3560G variant which we were trying to use for a small portable scrimmage box in a briefcase.

[FletcherS7]

I'm not arguing with the current solution, I just think it's old. And I've run into switch problems more often than not to be considered just a me issue. And the solution I don't think should be "buy more until you get one that works".

As I have mentioned on Chief
https://www.chiefdelphi.com/t/frc-blog-new-radios-at-2024-first-championship/447417/403?u=fletchers7

You can really use any IOS (or IOS XE) switch running 12 or later that is Layer 3 (Comes with IP-Base or better)
The California offseason fields ship with a Catalyst C3850-48U-S (from 2015, still in support) and a C3750E-48PD (from 2010)

[CJBuchel]

I sadly don't have those switches anymore. And the old code is no longer around as it was a precarious temp setup.
However the more I look into the old configuration the more I'm inclined to believe that I was mixing up the code changes of the switch and the code changes for the Linksys AP, which until the recent 2024 radio's/API was always a bit of a pain for us with OpenWRT commands.

While I do recall we had issues with the switches, perhaps it was just configuration issues and nothing code related.
So apology on that front, I likely came to the wrong conclusion.

I do have it's old configuration files though. If memory serves me right it was using IOS 13.
And the breaking config changes were due to ip http secure-server and vstack which had to be removed.

[FletcherS7]

And the breaking config changes were due to ip http secure-server and vstack which had to be removed.

Thankfully IOS just ignores invalid commands in the startup config (it will throw an error if you try and enter them into the running config though).
Both those commands are switch specific and not required for Cheesy Arena operation.

[CJBuchel]

Yea that must be it. I probably just saw the errors and removed them from the config. Then saw the OpenWRT commands not working and linked the two in my brain.

In any case while we're chatting I was curious on your thoughts on the alternatives I mentioned?

In an ideal world for me, I'd have a dropdown for 3 options. Cisco (default), Unify and then DHCP API.

It's not exactly a necessary change by any means. And as mentioned before the current change should work with any Cisco switch. But it would still be nice to have alternatives for non second hand switches. And make cheesy a lot more flexible for venues and events with the option of more modern hardware.

[FletcherS7]

Personally I think it adds unnecessary complexity.
But this is an open source project so go for it.

[justfishin1999]

As someone with a Cisco background, I lean toward continuing with Cisco gear, especially given its proven reliability and feature set. The 3560 and 3750 switches are tanks, and they’re relatively affordable now that they’re EOS/EOL. The 3850 series offers more modern capabilities and is also now EOL, though more recently than the 3560/3750 lines.

As I mentioned earlier, it would be nice to use something new. However, most switches the average person would consider purchasing aren’t true Layer 3. This switch must be L3-capable for the FMS to function with full advanced networking, and I’d imagine most folks aren’t eager to spend the money on a high-end, new L3 switch (the 9300 series runs $2000+). Buying used is dramatically cheaper, and in many cases, you can get both a primary and a backup unit for less than the cost of one new mid-tier switch. For example — the UniFi Switch Pro 24 runs around $400, plus you’ll need a controller. In contrast, a 3560G can be found for $50–$150 depending on the model, which gets you both a main and a backup unit for less than the cost of a single UniFi switch.

I did build a chooser into the settings to play around with the idea of supporting alternate hardware (it’s functional, but still in early testing), after being approached by another team trying to run their own FMS with a brand-new Aruba switch provided by their school district. Out of personal interest, I’m also testing an older L3 HP switch to evaluate broader hardware compatibility. I’m additionally working on using an ISR to handle routing and DHCP alongside a Layer 2 switch. While this adds some unnecessary complexity, it would allow almost any ISR with a g0/0 interface to be used without requiring code changes.

[CJBuchel]

I’m additionally working on using an ISR to handle routing and DHCP alongside a Layer 2 switch. While this adds some unnecessary complexity, it would allow almost any ISR with a g0/0 interface to be used without requiring code changes.

That sounds like a neat idea. I've not heard of an ISR before now, from a quick google it seems like just a switch to me. is there a particular benefit for them over a regular cisco switch?

I lean toward continuing with Cisco gear, especially given its proven reliability and feature set.

Also I agree that a second hand Cisco is still probably the best choice for people getting off the line. And the 3850 is a good shout for the switch people should try to aim for now. But I'm still onboard the idea of supporting other network setups that don't have to be second hand. Or could be flexible in allowing any setup. While still supporting and promoting second hand Cisco switches as the main method.

For us it seemed more like an evolution, we started with Cisco before moving to Unify to make venue setups easier through a UI, keeping everything all in one place and beefing up the setup for both the Field equipment and AV which use the same switch.