dpaa: implement SIOCGI2C for SFP modules in dTSEC - #2
Open
deGerard wants to merge 28 commits into
Open
Conversation
Add the MONO-GATEWAY kernel configuration for the NXP LS1046A-based Mono Gateway Development Kit. Based on GENERIC with OPNsense standard features, DPAA1 build infrastructure (empty file list for now), and early printf via QorIQ ns16550. Add the mono-gateway-dk.dts device tree covering the full SoC and board peripherals (DPAA1 FMan/QMan/BMan, I2C mux tree, GPIO, UART, eMMC, QSPI, SFP+ cages, and reserved memory regions). Add ARM64 I/O stubs (pio.h) for NCSW compatibility, EARLY_PRINTF qoriq type registration, CAAM RNG entropy source enum, and qoriq_therm build fix (remove spurious pci dependency).
Port the NetCommSW (NCSW) library from PowerPC to ARM64 for the NXP LS1046A. Adds ARM64 environment layer (xx_arm64.c, core_arm64.c, dflags_arm64.h), fixes big-endian portal access, cache operations, and integration headers for DPAA_VERSION 11 (FManv3/mEMAC). Includes enhanced hash table support (fm_ehash.c) for CDX PCD, MURAM batch-write fixes for ARM64 store behavior, QMan FQR RSS expansion, and BMan/QMan portal initialization changes. Enable device dpaa in GATEWAY config with NCSW file list.
Add the FreeBSD bus driver wrappers that bridge NCSW to the kernel device tree. BMan and QMan portal drivers handle multi-CPU initialization with explicit CPU binding (sched_bind unavailable during attach). FMan driver configures the frame manager hardware, reserved-memory regions (PFDR/FQD/FBPR via no-map DT nodes), and MURAM allocation. Portal common infrastructure provides shared init and cache-inhibited mapping for both BMan and QMan portals.
Add the main DPAA1 Ethernet driver using the NCSW FM_MAC API for mEMAC (LS1046A FManv3). Implements TX/RX via QMan frame queues and BMan buffer pools with scatter-gather support. Features: TX checksum offload (RPD+DTC), RX checksum validation with PCD parser, 128-FQID RSS via FMan KeyGen, deferred RX delivery via mbufq to prevent SMP hard lockups from nested NCSW_PLOCK, per-port BMan buffer pool with consumption model and in-flight cap. Includes mEMAC external MDIO controller for PHY management.
OH ports provide hardware packet processing pipelines for IPsec and WiFi offload via CDX. Port assignment matches Linux SDK: cell-index 3 (IPsec), cell-index 4 (WiFi). Cell-index 2 reserved for Host Command.
Firmware-based SGMII PHY used on the Mono Gateway (3x 1GbE ports). Key differences from standard PHY drivers: - No BMCR reset (kills firmware, darkens LEDs) - SGMII auto-negotiation via MMD VEND1 register - Status read from MIISTAT instead of BMSR
- qoriq_gpio: QorIQ GPIO controller support - gpiopower: GPIO-based power control fix - iicmux/pca954x: I2C mux devclass fix (use "iicbus" not "ofw_iicbus") - ofw_iicbus: FDT child enumeration fix - random_harvestq: allow CAAM hardware RNG as entropy source
TI INA219/INA226/INA234 I2C power monitor with sysctl interface for voltage, current, and power readings. Used on Mono Gateway for rail monitoring.
Class-based Enqueue/Dequeue Traffic Management for DPAA1 QMan. Provides hardware traffic shaping and scheduling via QMan CEETM channels.
Add callback hooks for bridge member add/remove and L2 address learning events. Used by CDX flow offload module to maintain hardware L2 forwarding tables.
Add callback interface for PF state lifecycle events (create, destroy, update). Used by CMM connection offload module to synchronize hardware flow tables with PF firewall state.
- qoriq_dw_pci: byte-swap CCSR register access for big-endian controller - pci_dw: lane configuration support for proper link training - DTS: enable PCIe3 for WiFi module (88W9098)
Process gpio-hog child nodes during gpiobus attach to configure pins at boot. Used on Mono Gateway for WiFi module enable (2R-enable), 3R-disable, and UART mux routing.
Add dtsec_rm_buf_free_external() for dpaa_wifi.ko to release BMan buffers from dtsec RX pools back to the correct UMA zone. Route M_PROTO1-tagged mbufs to if_transmit in QMan portal drain path.
fm_ehash: Rewrite ModifyMissNextEngine to match Linux SDK two-phase pattern (DROP→update→real action). Support KG (with CC_EN), PLCR (shared profile), and DONE engines. Remove verify-retry loop and CC tree copy propagation — single WRITE_UINT32 per word suffices with the two-phase approach. qman_portals: Add qman_portal_quiesce() to drain all per-CPU portal poll taskqueues. Called after FQR retire to ensure no in-flight QM_PORTAL_Poll references a stale callback from an unloading module. if_dtsec_rm: Add dtsec_rm_pool_rx_refill_bpid() for external callers (CDX dist FQ callback) to trigger BMan pool refill by BPID. dpaa_oh: Add distribution FQ callback registry (register/unregister/ lookup by BPID) for OH port consumers (WiFi, IPsec) to receive frames from CDX distribution FQs. Add MODULE_VERSION.
dpaa_oh: Add per-VAP distribution FQ registry so CDX can route WiFi download frames to hash-distributed FQIDs. Add dist fallback callback for frames with unrecognized BPIDs on OH dist FQs. Return EBUSY from dpaa_oh_enqueue when EQCR ring is full so callers can retry. Free dtsec buffers in error callback to prevent BMan pool drain. if_dtsec: Export dtsec_rm_pool_rx_refill_bpid for external callers (dpaa_wifi, dpaa_oh, CDX) that consume dtsec BMan pool buffers.
LAGG member ports are enabled via SIOCSIFFLAGS -> dtsec_if_enable_locked(), not dtsec_if_init_locked(). The MII tick callout was only started in the init path, so LAGG member ports never got periodic PHY polling. Without the tick, link state changes (cable unplug/replug) were never detected and RTM_IFINFO was never generated. Start callout_reset in dtsec_if_enable_locked() so member ports get the same periodic PHY polling as standalone ports. Also reprogram the hardware MAC address via FM_MAC_ModifyMacAddr() since LAGG changes the stack MAC via if_setlladdr() then cycles SIOCSIFFLAGS.
Add netgraph control message (cookie 20) that returns PPPoE session parameters: session_id, peer MAC, state, and hook name. Supports both single-session query (by hook name) and list-all-sessions mode. Used by CMM to detect active PPPoE sessions and extract the parameters needed to program hardware flow offload (session ID for PPPoE header insertion, peer MAC for L2 header construction).
Point FMCD_COMPILE_CMD include paths from $S/../../ASK/freebsd/fmlib/ to $S/../../opnsense-deps/fastpath/fmlib/. The headers are identical; this removes the build dependency on the standalone ASK repository.
Move the 5 fmlib ioctl ABI headers (fm_pcd_ioctls.h, fm_ioctls.h, net_ioctls.h, ioctls.h, integration_ioctls.h) from the external opnsense-deps repo into sys/contrib/ncsw/inc/ioctls/. This makes the kernel buildable without depending on a sibling repository. Merge FMCD_COMPILE_CMD into DPAA_COMPILE_CMD since it only added a single -I path; fman_chardev.c now compiles with the same flags as all other DPAA source files.
Wire SFP infrastructure into dtsec so 10GbE ports detect module insertion/removal via mod-def0 GPIO polling, read the SFP EEPROM for identification, control TX enable/disable, and report accurate link status via LOS GPIO instead of hardcoding link as always up. Module presence is polled at 1Hz in the existing tick callout. EEPROM reads are deferred to taskqueue_thread since I2C can sleep. Insert task re-checks module presence under DTSEC_LOCK to handle races with hot-removal during EEPROM read. Adds dev.dtsec.N.sfp_info sysctl for module vendor and part number. No DTS changes needed — sfp phandle, GPIOs, and i2c-bus are already defined in mono-gateway-dk.dts.
Use LINK_STATE_UNKNOWN for no-module state (instead of DOWN) so the sfpled driver can distinguish "no module" from "module present, no link". For copper PHY modules (RJ45 connector type 0x22), ignore LOS GPIO — these modules deassert LOS when powered up regardless of copper link negotiation status. Always report LINK_STATE_DOWN for RJ45 modules until a PHY driver can negotiate.
Add support for copper (RJ45) SFP+ modules with embedded PHYs. These modules use Clause 45 MDIO registers accessed via an I2C-to-MDIO bridge for PHY management, auto-negotiation, and link status polling. Two bridge protocols are supported with automatic detection: - RollBall (I2C addr 0x51, page 3, command-based) - Standard I2C-MDIO (I2C addr 0x56, SFF-8472) PHY probe runs on module insertion for RJ45 connectors, configures auto-negotiation (10G/5G/2.5G/1G advertisement with Extended Next Page), and polls PMA/PMD link status at 1Hz via taskqueue. Speed detection uses AN 10GBT_STAT for multi-gig and PMA/PMD CTRL1 as fallback. For copper modules without accessible PHY (autonomous operation), falls back to DDM RX_LOS polling for link state detection. Also fixes three GPIO issues that prevented copper modules from working: - TX_DISABLE pin direction must be set to OUTPUT explicitly - TX_DISABLE polarity inverted due to board hardware inverter - Module PHY stays inactive until TX_DISABLE is properly deasserted Verified at 10 Gbps (9.4 Gbps iperf3) with FS.com SFP-10G-T (AQR113C) and Intel X550 link partner over Cat6e cable.
Move ~900 lines of SFP management code (GPIO polling, EEPROM reading, RollBall I2C-MDIO PHY probing, auto-negotiation, link monitoring) from the dtsec MAC driver into the sfp_fdt framework driver. sfp_fdt now implements a full state machine (EMPTY → PROBE → PRESENT → PHY_PROBE/WAIT_LOS → PHY_AN/DDM_POLL → LINK_UP) that handles module detection, identification, and link management for any MAC driver with SFP+ cages. MAC drivers register a callback table (sfp_upstream_ops) via SFF_REGISTER_UPSTREAM() and receive module_insert, module_remove, link_up, and link_down events. dtsec is reduced to ~50 lines of callback implementations that update interface link state and store module info for sysctl reporting. Tested on hardware with both fiber (FLEXOPTIX DAC) and copper (FS.com 10GBASE-T with AQR113C PHY via RollBall) modules at 10 Gbps.
Change GATEWAY kernel config from UFS root to ZFS root (pool0/ROOT/default). ZFS is compiled into the kernel since booti direct boot has no FreeBSD loader to load modules. Disable kern.geom.label.disk_ident.enable so GPT labels work cleanly with ZFS. Remove printf from fman chardev ioctl default cases. ZFS GEOM probes all character devices with DIOCGMEDIASIZE (0x40086481) at boot, generating ~32 spurious "unknown ioctl" messages. ENOTTY is the correct response and the printf serves no diagnostic purpose.
Prefix every console line with a Linux-style [seconds.microseconds] timestamp. Brackets in default white, number in green via ANSI escapes. Uses microuptime() for the time source and emits directly to console hardware to avoid cnputc recursion. Both EARLY_PRINTF and normal console paths are covered.
This enables ifconfig -v to report SFP identity and, for modules that support it, diagnostic telemetry such as temperature, voltage, RX power, and TX bias.
deGerard
force-pushed
the
dtsec-sfp-telemetry
branch
from
April 10, 2026 10:15
68a3e38 to
a6a34f9
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This enables ifconfig -v to report SFP identity and, for modules that support it, diagnostic telemetry such as temperature, voltage, RX power, and TX bias.
I've tested with the following modules:
And it create data with ifconfig -v like:
plugged: SFP/SFP+/SFP28 10G Base-SR (LC)
vendor: FS PN: SFP-10GMSR-85 SN: G2430853700 DATE: 2024-11-01
module temperature: 30.84 C voltage: 3.27 Volts
lane 1: RX power: 0.66 mW (-1.82 dBm) TX bias: 6.43 mA