Skip to content

CM + SM Assembly

Documentation of combined Apollo assembly (splice plates et al)

Alternate configurations

TCDS jumpers

Bring-up Tests

This section describes the procedure for the bring-up tests, for the Rev2A Apollo hardware.

The very first test is to measure the resistances of some power sources, as described in the "Test 0" section below.

To conduct the next tests, the front-end panel must be assembled together with the board, which is described in the "Front-End Panel Assembly" section below. After the front-end panel assembly is complete, 5 more tests will be done by connecting the board to a power source. Please see below for descriptions of the 5 tests.

After the tests are finalized, the board should be booted using an microSD card. The last sub-section below describes information on configuring the microSD card for booting, before installing it into the blade.

As you go through the tests, please note down the results (e.g. base resistances, currents being measured) and save them into the Snipe IT database entry for the Service Module. You can find a results template to fill out here.

Test 0: Measuring Resistances

First test (before front-end panel assembly and any power-up) is to check the resistance values for several power sources. Using a voltmeter, please check the resistance values on the following sources:

  • 12V0
  • 3V3 STBY
  • 3V0
  • 4V5
  • 1V8
  • 3V3
  • 1V8Z
  • 3V3Z
  • 3V
  • 4V5Z

Please note that the labels for the power sources above are printed on the Service Module circuit board.

After the above checks are done, please check for the resistances across several fuses near PIM400, that will connect to the Zone-1 connector in the shelf, as explained below.

  • -48VA: The voltage difference between fuses F2 and F3, where F3 is the ground.
  • -48VB: The voltage difference between fuses F5 and F6, where F6 is the ground.
Measuring resistances between the two fuses for -48VA (left) and -48VB (right).

As you measure the resistances, please make sure to take note of them to the "Notes" section in the database entry. You can find a template to fill out here. If all resistance measurements are as expected (you can find the expected values in the Google document), test 0 is complete and you can proceed to front-end panel assembly below.

Front-End Panel Assembly

Before the other tests can be conducted, the Service Module board and the front-end panel (FP) need to be assembled together. The following needs to be done:

  • Remove the first two rows of the FP connector pins on the board, using a bolt cutter or something similar. After the pins are removed, add kapton tape to cover them.
  • Get two types of screws:
    • Two rounded head screws M2.5 x 5 mm 5306-11
    • Two rounded head screws M2.5 x 8.3 mm 5441-38
  • Screw the FP and the switches together using the two M2.5 x 5 mm screws.
  • Connect the switch plugin of the lower front panel latch to the correct side of the board.
  • If you have a front-panel board, attach it to the two connectors on the left leg of the Service Module, before attaching the board to the front panel.
  • To attach the board to the FP, unscrew the 8.3mm screw in the FP latches (if screws are already there). The board should go in between the space where the screw used to fill.
  • With the board now in place, attach the 8.3mm screw again, and you should have the board and the FP attached together.
Apollo Service Module before front-end panel is assembled. Connectors for the front-end panel switch and the front-panel board are also shown.
Apollo Service Module after front-end panel is installed. Here, the board is being inserted into the test shelf for the tests described below.

With the assembly complete, we can move on to the tests. It is recommended to do the following before moving on to the tests:

Check the power source: Making sure the blade is not connected to the power source, check the voltage of the source with a voltmeter, by powering it up. The potential difference you're measuring should be very close to 48 V. Turn off the power source after this check is complete.

Set the current limit: In the power source, set the current limit to be 0.5A for tests 1, 2, 3 and 4. For test 5 where Zynq will be installed, the current limit can be set to 1.0A instead.

Top left: Power supply configuration, please note that the current limit is set to 0.5A. Top right & bottom: Power supply connection to the test shelf.

Test 1

In this test, we'll connect the blade to the power source, and with nothing yet installed on it, we'll measure the current. We need to check two things:

  • The current value should be close to 0.08A
  • Check the voltage for the 3V3 STBY, it should be reading 3.3V

If the two bullet points check out, the Test 1 is concluded. You can turn off the power source.

Test 2

In this test, we'll add a jumper in the 12V EN slot (at the top right). With the jumper in place, we turn on the power source again, and check the following:

  • The current value should be close to 0.14A
  • The voltage for 3V3 STBY should be 3.3V
  • The voltage for 12V0 should be 12 V

If all bullet points check out, the Test 2 is concluded. You can turn off the power source.

Test 3

In this test, we will remove the jumper from Test 2, and install the IPMC to the Service Module. When installing the IPMC, make sure to include the serial number here. The serial number can be found using the QR code on the IPMC. The IPMC should be pushed in it's correct orientation to place, until it is fully connected.

The slot for the installation of OpenIPMC, highlighted in yellow. The locations of the two switches to force the OpenIPMC in place are shown by the red arrows.

Once these are done, turn on the power source, and the current value should be close to 0.15A. If this checks out, Test 3 is concluded. You can turn off the power source.

Test 4

In this test, we'll also install the ESM switch to the Service Module. To attach the ESM to the Service Module, we need the following:

  • Two 4.5mm x 6mm hex M2.5 standoffs
  • Two M2.5x6 screws (screwed from bottom)
  • Two M2.5x4 screws (screwed from top after the ESM is installed)
The slot for the ethernet switch, highlighted in yellow. The locations of the two screws are shown by the red arrows.

Once the ESM is screwed in, turn on the power source, and the current value should be close to 0.18A. If this checks out, Test 4 is concluded. You can turn off the power source.

Test 5

Note: Please increase the current limit of the power source to 1.0A, as the expected current will exceed 0.5A, the earlier current limit.

In this final test, we will install Zynq chip to the Service Module, and measure the current. Zynq chip should be pushed into the Service Module such that all three connectors are connected. Once this is complete, turn on the power source, and the current value should be close to 0.54A. If this checks out, Test 5 is concluded. You can turn off the power source.

The slot for the Zynq, highlighted in yellow. Zynq should be tightly pushed to its slot until the three connectors are connected tightly.

Once all the tests are complete, there are two stages left:

  • Installing the battery for the Real Time Clock (RTC)
  • Booting up the Service Module via an SDcard

RTC Setup

Before preparing the SM for booting, the battery for the real time clock (RTC) needs to be installed. This battery will power the RTC which is installed on Zynq.

Please install a CR1220 battery on slot J4 (for a rev2a SM).

Booting

This section describes how to boot the newly tested Apollo rev2a blades.

Configure the MicroSD Card

After the tests are finalized, for the first boot of the blade, we will use a microSD card with 32GB storage. The microSD card needs to be configured to contain two partitions, one partition for the boot files and the firmware, and another one for the filesystem. The SD cards should be set up according to the instructions here.

The 32GB microSD card to use.
Place the microSD card to the USB as follows, and plug it into a computer for configuration.

While filling up the first partition in the microSD card, you need to create a file called eth1_mac.dat. This file will specify the MAC address for the ETH1 network interface of the blade when it boots. This MAC address will then be used to determine the IP address of the SM within the EDF network, when it interacts with the DHCP server.

Within the eth1_mac.dat file, save the following value (you can find more information about the MAC address conventions here):

00:50:51:FF:10:<boardNumberInHex>

For example, if the board serial is 203, the last 8 bits of the address is CB.

After the microSD card is set up, place it in the slot on the bottom of the Service Module before booting. The next step is to configure the OpenIPMC in the Service Module such that the Zynq will boot using the microSD card. This is explained in the next step.

Configure the IPMC

The IPMC of the blade must be configured such that the booting is done from the recently installed microSD card (not the EMMC for now). To do this, one can connect to the IPMC via telnet, and write the desired configuration to the EEPROM. Please consult the instructions specified here for this.

Once the IPMC is configured and Zynq is booted from the microSD card, now you can proceed to configuring the EMMC, which is explained below.

Configure the EMMC

Once the board successfully boots, it must be set up so that it can boot up from the EMMC instead (i.e. bootmode 0). To do this, partitions and files on /dev/mmcblk0 must be configured. Similar to the microSD card configurations, you can follow the instructions here to set up the partitions, create the filesystems, and fill each partition with proper files.

After the EMMC is configured, configure the IPMC such that bootmode is set to 0, and restart the Service Module by executing restart on IPMC telnet CLI. Now, the blade should boot up using the files in EMMC instead. You can verify this once the blade boots up by doing:

# List all the mounts and look for EMMC mounts
$ mount | grep mmcblk
/dev/mmcblk0p2 on / type ext4 (rw,relatime)
/dev/mmcblk0p1 on /fw/SM type vfat (rw,relatime,fmask=0022,dmask=0022,codepage=437,iocharset=iso8859-1,shortname=mixed,errors=remount-ro)
If the EMMC partitions are mounted, you should be seeing that /dev/mmcblk0p1 and /dev/mmcblk0p2 device files are the ones being mounted. If you see /dev/mmcblk1pX instead, that means that the SD card partitions are mounted.

Installing HeatSinks on ZYNQ

There are three parts that make up the Heatsink: The blue Aluminum heatsink itself, a metal clasp to attach it to the Zynq, and a rubber support. To begin set-up, carefully wrap the rubber hold around the Zynq, making sure that the protruding lip hangs on to the Zynq itself. You can make sure it is properly attached by gently pulling on the rubber and confirming that it stays firmly attached. Next, center the heatsink such that the the middle of the heatsink is aligned with the two loops on the rubber support. This ensures that the sink is properly aligned with the airflow coming through the ATCA shelf. Finally, orient the clasp along the middle of the heatsink such that it snaps below the two black loops on the rubber support. Try doing so by hooking the metal clasp on one side first. Once secured, gently push down to attach it on the other side.

Note that after snapping the metal clasp securely on both sides, make sure it is not in contact with any of the capacitors or other electronics around the Zynq. The configuration should be stable and light tugs should not disrupt the structure of the heatsink with its rubber support and metal clasp.

Top: Individual parts composing Heatsink. Top left is the Heatsink, top right is the rubber support, and bottom is the metal clasp. Bottom: Heatsink set-up on a Zynq.

Programming the CPLD

As the final step, the Complex Programmable Logic Device (CPLD) should be programmed, so that the JTAG connections between Zynq and the Command Module function properly. To do this, you can clone the SM_Rev2_CPLD_FW repository. You will also need to download the top.jed file from the latest release, which can currently be found here for v0.0.1.

Before programming the CPLD, to enable XVC JTAG between Zynq and CPLD, you'll need to set the SERV.CPLD.ENABLE_JTAG register to 1. You can do that by SSHing to the blade of interest, and launching BUTool:

# SSH as cms or atlas user
ssh cms@apollo2xx-1

# Launch BUTool CLI and write 0x1 to register
BUTool.exe -a

> write SERV.CPLD.ENABLE_JTAG 1

Once this is done, program.py script can be executed to program the CPLD:

# Clone the repository
git clone https://github.com/apollo-lhc/SM_Rev2_CPLD_FW.git

cd SM_Rev2_CPLD_FW/
./program.py --xvc_ip apollo2xx-1.edf.lo --jed_file /path/to/top.jed

Note that program.py needs to be provided with the IP address (or domain name) of the blade, and the path to the installed .jed file. The script also takes additional arguments:

Please note that if you're running this on Tesla machine, you do not need to modify the following arguments, you can just use the defaults.

  • --VIVADO_SOURCE: Path to the Vivado installation on the machine
  • --ISE_SOURCE: Path to the ISE installation on the machine

You can check out the output of ./program.py --help for the full list of available options.

Once the programming is done, please write 0 to SERV.CPLD.ENABLE_JTAG register to disable XVC JTAG.

Setting Zynq Time

Please follow the instructions here to set the correct time on the Zynq SoC.

Assembly

Please look at the disassembly instructions before the assembly instructions if you have a partially assembled board.

  1. If completely un-assembled, reference the bring-up instructions for the SM bring-up assembly. There should be no front panel or front panel board on the SM at this time.

    • (Rev1) With no splice-plates on, slide the CM into the SM until the board-to-board connectors mate
    • (Rev2) Install splice plates on the CM and then following the silkscreen outline on the SM, place the CM down and slide it into the SM untilt the board-to-board connectors mate
    • (Rev1) Install both splice plates with 3 SM and 3 CM screws on each splice plate.
    • (Rev2) Install two screws connecting the splice plates to the SM (2 screws per splice plate)
  2. Install the front panel board on J40,J46 (rev1) J40,J46,NJ207 (rev2,rev2a)
  3. Remove the M2.5 screws from the SM front panel ATCA switches (in closed position)
  4. Slide the SM front panel onto the SM aligning it with the SM board holes for screws from step #5. Slight adjustment of the front-panel board might be needed to fit LEDs and USB port into front panel slots.
  5. Install the ATCA switch M2.5 screws.
  6. Plug the lower ATCA switch's 3pin connector into SM (J7)

Disassembly

  1. (Rev1 only) Remove the front and back covers (link?)
  2. Close the ATCA switches to prevent the springs from popping out.
  3. Unplug the lower ATCA switch's 3pin connector from the SM (J7)
  4. Remove any screws connecting the CM's front panel plate to the SM's front panel.
  5. On the back-side of the blade, remove the two M2.5 x 8.3mm screws connecting the SM to the FP switches.
  6. Carefully remove the SM front panel (includes ATCA switches).
  7. Re-install the M2.5 screws from step #5 into the now separate FP board
  8. Remove the SM's front panel board connected to J40,J46 (rev1) J40,J46,NJ207 (rev2,rev2a)
    • (Rev1) Remove both splice plates by removing 3 SM side and 3 CM side screws on each splice plate
    • (Rev2) Remove SM side screws (2 per splice plate)
  9. Slide CM away from the SM to disconnect board-to-board connectors and then lift CM up and away