Netra CP2300 cPSB Board Installation and Technical Reference Manual

This chapter contains the following sections:

• Section 5.1, Summarized Physical Description
• Section 5.2, Detailed Description
• Section 5.3, CPU and Main Memory Subsystems
• Section 5.4, Bus Subsystems
• Section 5.5, System Input/Output
• Section 5.6, System Management Controller
• Section 5.7, Resets
• Section 5.8, Power Subsystem
• Section 5.9, CompactPCI Interface
• Section 5.10, Interrupts
• Section 5.11, Chip-Select PLD Registers

---

5.1 Summarized Physical Description

The Netra CP2300 board is a 6U-sized cPSB circuit card with CompactPCI connectors J1 (labeled as J9 on the board) and J2 (J10) for PCI, and J3 (J13) and J5 (J14) for I/O. The CompactPCI J4 connector is not fitted to the board. See FIGURE 5-1 and FIGURE 5-2 for top and solder-side views of the board.

 FIGURE 5-1 Netra CP2300 Board Layout

 FIGURE 5-2 Typical Netra CP2300 Board - Solder Side

The Netra CP2300 board functions as a cPSB node board. The board has the following I/O access:

• One serial port on the front panel.
• Through the backplane: a transition card must be connected to provide rear connector access for two serial ports, two optional switch-selectable Ethernet ports, a switch-selectable EIDE connector, and a USB port. Sun Microsystems offers a compatible Netra CP2300 cPSB transition card (part number, XCP2300-TRN, 375-3134-xx). This card is recommended for use with the Netra CP2300 board, but OEM developers may design their own transition card instead.
• Two PMC card interfaces can accept two PMC I/O cards.
• The Netra CP2300 transition card has two PIM interfaces that can accept two PIM cards that can bring the PMC ports out to its panel at the rear of the enclosure. See Section 5.4.2, PMC and PIM Interface for further details.

The front panel of the Netra CP2300 board has two PMC module slots, a serial port, and the following status LEDs:

• ABORT - push button (XIR)
• RESET - push button (POR)
• USER - yellow
• READY - green
• Blue LED for hot-swap

 FIGURE 5-3 Front Panel of a Typical Netra CP2300 Board Assembly With Heat Sink

---

5.2 Detailed Description

A simplified schematic diagram is shown in FIGURE 5-4.

 FIGURE 5-4 Netra CP2300 Board Functional Block Diagram

The L2-cache is integrated into the UltraSPARC IIi processor package. This processor is supported by SDRAM memory that is soldered onto the board and is also available in plug-in module form.

Apart from incoming interrupts, the processor handles all I/O through its built-in
66 MHz, 32-bit PCI bus interface. This interface is used to connect to a Sun Advanced PCI Bridge (APB) that services two 33 MHz 32-bit downstream interfaces, PCI bus A and PCI bus B.

PCI Bus B connects the APB to two Davicom Ethernet packages and the South Bridge. In addition, The PCI bus A from the APB connects to each of two 33 MHz, 32-bit PMC interfaces on the host board. See Section 5.4.2, PMC and PIM Interface for more details.

The SMC connects to:

• The on-board I²C bus, enabling it to communicate with sensors and controls.
• The User IPMI bus, enabling user management of other entities in the system. Peripheral hot-swap control is also enabled through this path.
• The power subsystem.

The SMC controls the startup of the board, because it activates the power module and controls the system reset signals. In addition, it handles hot-swap signals from the CompactPCI backplane, for example: HEALTHY and BD_SEL. See Section 1.3, Hot-Swap Support for more information on hot-swap.

---

5.3 CPU and Main Memory Subsystems

This section describes the UltraSPARC IIi processor and additional memory on the Netra CP2300s. FIGURE 5-5 shows the UltraSPARC IIi interfaces.

5.3.1 UltraSPARC IIi Processor

 FIGURE 5-5 UltraSPARC IIi Interface

The Netra CP2300 board uses the UltraSPARC IIi 650 MHz processor. The processor is housed in a 370-pin ceramic pin grid array (PGA) package. It typically dissipates no more than 18 W at 650 Mhz (the entire Netra CP2300 board may dissipate 25 W).

The UltraSPARC IIi processor is directly connected to the board SDRAM through a 72-bit (64-bit DATA plus 8-bit ECC) path. Dual address buses reduce capacitive loading and increase the memory density beyond that of unbuffered devices.

The CPU connects to the APB by means of a 32-bit, 66 MHz PCI interface which the APB in turn translates to two downstream 33 MHz PCI buses.

The UltraSPARC processor begins execution from a fixed image in a PROM that lies on the XBus. The processor accesses this XBus in a boot path that automatically includes the APB, 1535 Bridge, and XBus.

Processor resets are received from the system management controller (SMC). See Section 5.7, Resets and FIGURE 5-16 for more details.

The various interrupts on the board are encoded by the I-chip2 to appear at the UltraSPARC IIi processor as 6-bit parallel data. See Section 5.7, Resets for more information.

The processor I/O is run at a fixed VDDIO of 3.3V but the core voltage, VDDCORE, is adjustable and configured according to CPU type, typically in the range of 1.3 V to 1.9 V.

JTAG/Test signals are available for use in boundary scan diagnostics.

5.3.2 Memory Address Mapping

The UltraSPARC IIi L2 cache megacell reserves a 4 GB region for cacheable main memory. The memory databus width and the module databus width are of equal size (64-bit data plus 8 bit ECC) so memory modules can be installed in mixed sizes.

The UltraSPARC IIi Address Data Generation Logic (ADGL) logically maps modules according to their size, rather than their physical location. The largest sizes are mapped to the lowest address ranges. Where modules of identical size are present, the lower slot number is mapped to the lower address range.

FIGURE 5-6 shows a memory mapping example.

 FIGURE 5-6 Memory Mapping Example

5.3.3 SDRAM Memory

 FIGURE 5-7 SDRAM Memory Interface

The UltraSPARC IIi 650 MHz processor connects directly to the memory with a 72-bit (64-bit DATA plus 8-bit ECC) data bus. The memory can be either on-board SDRAM only or can include additional SO-DIMM memory modules.

Each module is equipped with a serial EEPROM device containing 256 bytes of presence detect data.

The memory block, whether on-board or a module, provides non-volatile serial memory to enable the Serial Presence Detect function. This memory can be interrogated by the SMC through the I²C interface.

5.3.4 Memory Components

This section describes additional memory available on the Netra CP2300 boards.

5.3.4.1 System (Boot) Flash Memory

The system flash usually resides in 1 Mbyte of space (see Section 4.7.1, Exchanging the System and User Flash Memory Devices for more information). It contains Common Operations and Reset Environment (CORE) firmware, Comprehensive POST, and OpenBoot PROM boot code. The system flash may be upgraded by running a program out of OpenBoot PROM or executing a Solaris software script. If the system flash becomes corrupted, contact your nearest Field Application Engineer.

5.3.4.2 User Flash Memory

The board is usually equipped with 7 Mbyte of user flash memory (see Section 4.7.1, Exchanging the System and User Flash Memory Devices for more information). You may use the flash memory for various purposes such as storage for RTOS, user data storage, OpenBoot PROM information, or to house drop-ins. Drop-ins simplify customizing a system for the user.

A DIP switch SW503 and an SMC configuration block setting determines whether the user flash is bootable during OpenBoot PROM boot and whether it is write enabled (see Section B.5, DIP Switch Settings).

5.3.4.3 NVRAM

The Netra CP2300 board uses the I²C serial EEPROM to save configuration variables.

These boards uses a time-of-day (TOD) I²C device and a 8 Kbyte serial EEPROM. These components provide:

• A time-of-day (TOD) real-time clock, which features 56 bytes of non-volatile SRAM. The real-time clock counts seconds, minutes, hours, day the of month, month, day of week and year, with leap year compensation. Address and data are transferred serially through the 2-wire bi-directional bus.
• 8 Kbyte storage for environment variables, user modifiable. The Ethernet address and Host ID are stored in the system flash. These values are copied to the I2C SEEPROM when the system is powered-on.

5.3.4.4 Serial I²C EEPROM

This device stores the OpenBoot PROM configuration variable settings, the board MAC address, and the Host ID in a removable serial EEPROM that is accessible through the I²C bus.

5.3.4.5 FRU ID I²C EEPROM

The FRU ID I²C EEPROM chip stores manufacturing-related information of the Netra CP2300 board. This information is useful only when the Netra board is being serviced.

See Appendix C for details on accessing the FRU ID information for the board.

---

5.4 Bus Subsystems

There are three internal PCI buses on the board. One of the internal PCI buses, PCI bus B, is bridged to the slower ISA-speed XBus.

 FIGURE 5-8 Netra CP2300 PCI Bus Interface, 33MHz CompactPCI Bridge

5.4.1 APB PCI Bus Interfaces

The UltraSPARC IIi CPU has an integrated 32-bit/66 MHz PCI bus interface. The Advanced PCI Bridge--acting as a North Bridge--splits this bus into two
32-bit/33 MHz PCI buses. Of these, the PCI A bus connects to the PMC slots and the PCI B bus connects to the onboard I/O devices.

5.4.2 PMC and PIM Interface

The PCI mezzanine card (PMC) interface is defined by IEEE and PICMG standards:

• IEEE P1386, presently in draft form, defines the Common Mezzanine Card (CMC) mechanical profile (common because this definition, for example, can also apply to a VME rendering)
• P1386.1 defines the PCI electrical interface through its connectors Pn1/Jn1 through Pn4/Jn4
• PICMG 2.3 R1.0, PMC on CompactPCI Specification maps the PMC signals from the Jn1 through Jn4 connectors on the CompactPCI board through the CompactPCI backplane connections. In the case of the Netra CP2300 board, these connections are routed to the CompactPCI J3 (labeled as J13 on the board) and the J5 (J14) connectors.

The PMC interface enables you to use Independent Hardware Vendor (IHV) PMCs to implement additional I/O from the host at the system integration level.

The Netra CP2300 cPSB transition card provides two slots for PIM cards. A PIM card enables rear I/O functions when paired with a PMC card installed on the front panel of the board.

A PIM must have a corresponding PMC in the front slot because the PMC board performs an adapter function between the PCI bus A and the user I/O signals passed through a CompactPCI backplane connector to the PIM slot on the transition card. When a PMC that has a front-panel connector is used with a PIM, jumpers are typically set to disable its front I/O operation.

FIGURE 5-9 illustrates the interconnection between PMC and PIM slots for installed PIMs. The numbers in parentheses show how the connectors are labeled on the Netra CP2300 node board and transition card.

 FIGURE 5-9 PIM Installation Configuration

FIGURE 5-10 shows the PMC B connectors and card attachment to a Netra CP2300 board. There is a second PMC slot, PMC A, adjacent to the first (see FIGURE 5-11).

 FIGURE 5-10 Data Paths in PCI Mezzanine Module Interface on Host Board

For the PMC B slot, the APB on the Netra CP2300 board supplies PCI bus signals to PMC B connectors Jn1 (labeled as J6 on the board) and Jn2 (J12). The PMC card logic decodes its specific I/O interface, which it makes available at the front panel.

The 64-pin PMC Jn3 connectors are not fitted to the Netra CP2300 board. These connectors are specified for expansion for 64-bit PCI (it carries the upper 32 bits), which is not provided. A 64-bit capable PMC card can function in these slots but its bus interface is constrained to 32 bits.

The PMC B Jn4 (J11) connector is specified for user I/O and carries PMC signals to the CompactPCI J5 (J14) backplane connector. If a transition card is installed, this CompactPCI J5 (J14) connector I/O is conditioned by an IHV-supplied PIM card to provide the matching I/O on the enclosure back panel. Its backplane I/O is routed to the CompactPCI J5 (J14) connector.

In the case of the PMC A slot on the Netra CP2300 board, the PMC A Jn1 (J5) and the PMC A Jn2 (J8) connectors are similarly attached to the APB, but the user I/O from the PMC A Jn4 (J7) connector is routed out of CompactPCI J3 (J13) backplane connector.

See FIGURE 5-11, TABLE 5-1, and TABLE 5-2 for more information about the Netra CP2300 board PMC and CompactPCI backplane connector labelling.

 FIGURE 5-11 PMC Connector Interfaces on the Netra CP2300 Board

TABLE 5-1 lists how the PMC slot connectors are labeled on the board, and TABLE 5-2 lists how the CompactPCI backplane connectors are labeled.

5.4.3 I²C and IPMI Channels

The SPARC domain I²C controller is located on the ALi 1535D+ South Bridge chip. The SMC also has two on-board I²C channels used for IPMI and one external I²C controller to access on-board I²C devices.

I²C communication is used:

• To implement the IPMI interface for system management.
• To provide a means of performing local environmental monitoring which tracks--and controls where appropriate--local board or chassis "housekeeping" functions. These functions include: monitoring of temperature, FRU ID, MAC address, and memory type and size.
• To provide an interface for user monitoring and control, for example chassis temperature or fan operation.

The I²C paths are shown in FIGURE 5-12.

Each I²C device on the board uses common addressing pins. The devices are distinguished by the internal device ID and selectable address pins. All I²C devices are supplied from IPMB or early power before backend power is established whenever possible (see Section 5.8, Power Subsystem for further details).

 FIGURE 5-12 Netra CP2300 Board I²C and SMBus Architecture

---

5.5 System Input/Output

 FIGURE 5-13 Netra CP2300 Board and Transition Card I/O Interfaces

FIGURE 5-13 shows the I/O for the Netra CP2300 board. The I/O functions can be categorized into four groups which are described in the following sections.

5.5.1 Front-Panel I/O

FIGURE 5-14 illustrates the indicators and I/O connectors on the Netra CP2300 front panel. The Netra CP2300 front panel connectors, buttons and LEDs are described below:

• Two peripheral mezzanine card (PMC) I/O bezels
• A serial connector (DIN8) which is also connected to serial port A of the 16552 UART on the board
• ABORT - An abort push button; passes an XIR signal to the SMC
• RESET - A reset push button; passes an Power-on-Reset (POR) signal to the SMC
• USER - A yellow LED that is defined by the user. The default function for the LED is to signal that the board is at an OK status. The colors are controlled by SPARC by way of the SMC. See the Netra CP2300 cPSB Board Programming Guide (817-1331-xx) for information on programming the user LED. You can download this document at:

http://www.sun.com/products-n-solutions/hardware/docs/CPU_Boards/

• READY - A green power LED, sourced from the power module and controlled by the SMC.
• A blue LED for hot-swap status, sourced from the SMC.

 FIGURE 5-14 Netra CP2300 Board Front Panel

5.5.2 PMC Interface

The host board includes two PMC front-panel I/O cutouts to enable attachment of up to two PMC expansion cards. When installed, these cards access a PCI bus through compatible connectors provided on the host board. See PMC and PIM Interface.

5.5.3 Backplane I/O

Most of the I/O channels to or from the board are passed to the CompactPCI connectors: J3 and J5 (which are labeled J13 and J14 respectively). These channels are accessible from external connections on a transition card connected at the rear of the backplane. The CompactPCI J4 connector is not populated on these host boards to prevent contention with H110-compliant backplane signals. Contact assignments for these connectors are shown in Section B.4, Backplane Connectors. For location of the connectors, see FIGURE 5-1.

5.5.3.1 CompactPCI J3 (J13) Signals

The user-defined PMC A I/O signals on the Netra CP2300 cPSB board provides matching I/O interfaces on the enclosure backpanel through the PIM cards installed on the Netra CP2300 transition card. The PMC A I/O signals are routed through board's CompactPCI J3 connector (labeled as J13 on the board) to the transition card's CompactPCI rJ3 connector (labeled J1 on the transition card).

5.5.3.2 CompactPCI J5 (J14) Signals

The following signal sets pass through the CompactPCI J5 backplane connector (labeled as J14 on the board) to connect to an external interface connector on the transition card:

• Two TP Ethernet channels from the PHYs
• Two serial channels
• A USB channel
• An I²C channel
• PMC B I/O

The PMC B I/O signals are routed through board's CompactPCI J5 connector (labeled as J14 on the board) to the transition card's CompactPCI rJ5 connector (labeled J2 on the transition card).

---

5.6 System Management Controller

The System Management Controller (SMC) subsystem is one of the most important components of the system board. This subsystem provides a variety of service functions related to assuring availability of the system. These functions contrast with the board functions that execute applications.

The SMC consists of a small microcontroller with an SRAM for a software stack and nonvolatile memory for program storage and data logging. The SMC is modular in character, but is physically embedded into the circuitry of the Netra CP2300 board. FIGURE 5-15 shows its functional relationship with the system.

 FIGURE 5-15 System Management Controller Interface

The SMC hardware and firmware implements the functions of system management and hot-swap control.

The SMC controls the on-board CompactPCI interface for the hot-swap process.

In performing these functions, the design maintains conformance with the PICMG CompactPCI core specification and the PICMG CompactPCI hot-swap specification: See Appendix D for references to these documents. The main features that are supported by the SMC subsystem are:

• Coordinates and controls local resets on the system board during power-on reset, watchdog timeout, software initiated reset, and as a result of user intervention such as pushbutton reset or backplane reset.
• Supports a command and communications interface between the UltraSPARC IIi host processor and the SMC microcontroller by means of an interconnecting XBus. This interface accommodates a command suite from the UltraSPARC processor to the SMC and supports bidirectional interrupts between the UltraSPARC processor and the SMC.
• The board responds to the CompactPCI RST, HEALTHY, and BD_SEL signals from the active system controller (provided that this host provides HA hot-swap functions through the CompactPCI bus).

• Implements a two-level watchdog timer for the SMC processor and for the host processor.
• Supports three I²C ports. One of the I²C ports carries an IPMI bus that is routed through CompactPCI J1 backplane connector (labeled as J9) to enable communication with other SMCs in the system. The second I2C port carries IPMI signals to the CompactPCI J2 connector (J10). The third I²C port serves a multiplexer that splits its input into four channels:
• One channel provides communication with the FRU ID and motherboard (for example, temperature or FRU information)
• A second channel is for user functions and transition card FRU ID and is passed to the CompactPCI J5 connector (J14)
• A third channel is for ADM 1026 system monitoring
• A four channel is for time-of-day (TOD), SPD, and other on-board devices

• Communicates with the system controller using the IPMI protocol over a backplane link, in accordance with the PICMG CompactPCI hot-swap specification. The SMC responds to commands from the active Baseboard Management Controller (BMC) when the host board is installed in a peripheral slot.
• Enables local a local environmental monitoring management scheme that monitors--and controls where appropriate--local board or chassis "housekeeping" functions through the on-board I²C interface. Such functions include (see also Section 4.5, Environmental Monitoring Support at OpenBoot PROM):
• Temperature sensing
• Power supply voltage sensing
• Power supply module on/off control
• Memory module and on-board SPD detection

• Supports flash update - the SMC firmware supports external update of its flash PROM (see Chapter 4).
• Acts as a peripheral management interface, which includes:
• IPMI communications with Baseboard Management Controller
• Handling of hot-swap (HEALTHY/BD_SEL) related signals
• Receiving system and CompactPCI reset events to generate local board reset

For full details on SMC and reset information, refer to Chapter 4.

5.6.1 Watchdog Timer

In the Netra CP2300 board, the SMC implements a two-level watchdog timer. The host-SMC command interface defines communication between host and SMC. The host and the SMC constantly communicate with each other when the watchdog timer is enabled. The SMC monitors the heartbeat of the CPU processor host. The heartbeat is sent in the form of a reset watchdog timer that is sent from the CPU to the SMC. The watchdog timer must be programmed to ensure that it does not get too close to the expiration. There should be some time accounted for the latency overhead or any unexpected event that may delay transmission of the heartbeat. For full details on programming the watchdog timer, refer to the Netra CP2300 cPSB Board Programming Guide (817-1331-xx).

The two levels of the watchdog timer are as follows:

• Countdown register timer (16 bits, 100 msec. resolution)
• Pre-timeout timer (1 sec. resolution)

The pre-timeout timer can only start if the countdown register timer has expired or has been set to zero. The two timers cannot run simultaneously.

The two watchdog timers are enabled by messages sent over the host-SMC command interface using the set watchdog timer command. The commands to enable the host-SMC command interface for watchdog timer functionality are:

• smc-reset-wdt
• smc-set-wdt
• smc-get-wdt

The uses of these functions are shown in TABLE 5-3.

---

5.7 Resets

This section provides details on resets for the Netra CP2300 board.

 FIGURE 5-16 Simplified Reset Paths

Parts of the board are powered by early power before the SPARC domain receives power (backend power). For more information, see Section 5.8, Power Subsystem.

The SMC receives its power through the 5 volt standby (IPMI) power. However, if early power is present, the SMC switches itself to receive power from early power.

Once the SMC is out of reset and initialized, the SMC powers the CPU SPARC by turning on the backend power (power module). After the power module successfully powers-on, it will assert a POWER_OK signal. When the POWER_OK signal is asserted, the SMC will remove the CPU SPARC resets, which will allow the CPU to boot.

Note that:

• When the SMC is reset, the whole system is reset.
• When the CPU is reset, the CPU I/O is reset.

 FIGURE 5-17 Simplified CPU Subsystem Reset Architecture

---

5.8 Power Subsystem

FIGURE 5-18 shows a simplified schematic diagram of the power subsystem. This subsystem can power the board to support a hot-swap environment.

 FIGURE 5-18 Power Distribution Block Diagram

The Netra CP2300 board sequences power in two time-separated domains:

• Early Power domain
• SPARC Power domain--this is the Backend Power in the PICMG hot-swap specification.

Early power is applied to the board from backplane long pins (LP in the figure) as the board is inserted. Early power current flows to board subsystems:

• Power block - supplies precharge current to the CompactPCI bus interface components.
• SMC - needed to control logical state of the CompactPCI interface circuits as they are connected.
• IPMI/I²C subsystems - needed for management/monitoring functions at this stage; I²C power also extends to the transition card.

5.8.1 Power Module

FIGURE 5-19 shows a schematic diagram of the power module. This subassembly is integrated with the Netra CP2300 board.

 FIGURE 5-19 Power Module Interface

This subsystem performs the following functions:

• Generates Vp, the CompactPCI hot-swap precharge bias voltage using early power
• Generates VDDCORE, the UltraSPARC processor core voltage supply
• Controls and gates 5V, 12V, 3.3V and -12V
• Automatically shuts down in case of overcurrent or over/under voltage
• Asserts the PWR_MOD_OK signal

The power module is controlled by the SMC and the power on/off signal. Functions controlled include core voltage, and backend on or off state. There are also automatic controls within the power module, for example, overcurrent shutdown, and voltage regulation.

The power module has a DIP switch with preset default settings. These switches are for factory use only (see FIGURE 5-20 for location). You must not change DIP switch settings for positions 1 though 5. Switch position 6, however, can be used to set the firmware userflash and system flash settings. See Section B.5.1, SW3 DIP Switch for more information.

 FIGURE 5-20 DIP Switch SW3 Settings on Power Block

5.8.2 Early Power and IPMI Power

If the system power for the backplane fails, the SMC can use IPMI power, typically supplied from an uninterruptible power supply (UPS), instead of early power from the cPSB backplane. The backplane is provided with IPMI power pins for this purpose. FIGURE 5-21 shows the circuit arrangement that selects between these power sources.

 FIGURE 5-21 Selection Between Early Power and IPMI Power

5.8.3 Transition Card Power Distribution

FIGURE 5-22 shows the power rail routing to the transition card. The Netra CP2300 cPSB transition card is powered from the Netra CP2300 board rather than directly from the backplane. The transition card must always be connected to the backplane before the chassis is powered. Always install the transition card before the Netra CP2300 in the chassis. For details on using a transition card, see Chapter 2.

 FIGURE 5-22 Transition Card Power Supply Routing

---

5.9 CompactPCI Interface

This section provides information on CompactPCI and the Netra CP2300 board interface requirements specifications and CompactPCI signal interface.

5.9.1 CompactPCI Interface Requirements

TABLE 5-4 lists the requirements for Netra CP2300 boards as defined by the
PICMG 3.0 CompactPCI and Netra CP2300 board design requirements specifications:

5.9.2 CompactPCI Signal Interface

This section lists the CompactPCI connector power signal interfaces.

---

5.10 Interrupts

The Netra CP2300 board interrupts are listed in TABLE 5-6. These are processed and encoded by the I-Chip2 ASIC. This device assigns equal priority to all interrupting devices. When two devices need servicing at the same time, the I-Chip prioritizes using its internal round-robin scheduling scheme. The resultant vector is passed to the processor as a 6-bit parallel word. The ultimate interrupt priority is resolved in the UltraSPARC IIi processor.

---

5.11 Chip-Select PLD Registers

TABLE 5-7 lists the chip-select programmable logic device (PLD) registers.

Netra CP2300 cPSB Board Installation and Technical Reference Manual

816-7186-12

Copyright © 2004, Sun Microsystems, Inc. All rights reserved.