Get Even More Visitors To Your Blog, Upgrade To A Business Listing >>

Creating an FPGA-Based Low-Cost Imaging System

Creating an FPGA-Based Low-Cost Imaging System

Not all imaging systems need to be expensive. Solutions can be created using cost optimized FPGA and CMOS image sensors directly.

IntermediateFull instructions provided3 hours3,969

Things used in this project

Hardware components

Digilent Arty S7-50
× 1
Xilinx TDNext 1.26Mpixel Pmod Camera Kit
× 1
ZedBoard MicroZed 10-inch Touch Display Kit
× 1

Software apps and online services

Xilinx Vivado HLx

Story

At completion of the integration time each pixel is read out as either a 8 bit or 10 bit pixel. This pixel value is referred to as a RAW8 or RAW10 pixel. To recreate a color image, the values of surrounding pixels which contain pixels of different wavelengths are combined using a de-Bayering algorithm.

Vivado Build

The first thing we need to do is create the Vivado platform, this will receive the images from the TDNext Pmod.

To create the block diagram we will be using mostly IP cores from the Vivado library however we will use a camera interface block and a output block from the Avnet library which is available here.

The first step is to install the board definition files, this enables Vivado to understand the configuration of Arty S7. You can download the board definition files from here.

Once downloaded these files should be installed in your Vivado directory under the following path:

/Vivado//data/boards/board_files/

This will allow you to then select the Arty S7 board as your target board for the creation of a new Vivado project.

With the board installed, the next step is to create a new project, block diagram and create the MicroBlaze system.

The fastest way to create a MicroBlaze system is to follow the steps in the video below

How to create a MicroBlaze system in the Arty S7

With the MicroBlaze system up and running, the next step is to add in the video processing pipeline. The processing chain will use the following IP blocks

  • CAM Interface – This interfaces with the TDNext video interface
  • Video to AXIS – This converts parallel video to AXI Streaming format
  • Sensor Demosaic – This converts the RAW pixel values which represent either R, G or B into a 24 bit RGB format
  • Video Timing Generator – This generates the video timing signals for the output format
  • AXI Stream to Video Out – Converts AXI Stream to parallel video
  • ZED ALI3 Controller – IP module to drive the 10 inch touch display
  • AXI IIC – Attached to MicroBlaze this will be used to configure the imager
  • AXI UART – Attached to MicroBlaze used to report system status to user

If we are using the Pmod VGA we do not need to use the ZED ALI3 Controller IP block.

Before we can add in the Zed ALI3 and CAM Interface we need to reconfigure the IP core to be able be included in Spartan 7 designs. We do this from the IP catalog view, selecting the desired IP cores and clicking edit IP in packager.

This will open a new project and enable you to select the comparability tab and add in support for the Spartan 7 device. Repackage the design and update the IP lib in the Vivado project.

Once the IP have been upgraded to support the Spartan 7 we are then in a position to complete the design. The complete bock diagram should look as below. It is available on my GitHub Account for a closer look.

Unlike previous examples based upon heterogeneous SoC where used an external frame buffer. This example will not use VDMA to read and write from an external frame buffer, such a approach requires a different configuration of the AXIS to Video and VTC.

Normally the AXIS to video is configured as the master and the VTC is not controlled. However, in this approach the AXIS to video is configured as a slave and the VTC generator clock enable is controlled.

This approach allows the AXIS to Video IP module to control the timing of syncs by enabling and disabling the VTC, so they match the timing of the Syncs in the processing pipeline.

Within the Axi Stream the start of frame is indicated by the use of TUser and the end of line is indicated by the use of TLast.

Key customization for the IP blocks are :-

Video In to AXI 4 Stream

Sensor Demosaic Settings

In the design I have also added in several Integrated Logic Analyzer (ILA) to enable internal monitoring of the status of the system and debugging.

The total utilization of the Arty S7-50 when the project is completed is shown below.

We can use the additional resources to implement image processing algorithms using HLS is necessary. If we want to save resources, we can use the minimum footprint for the MicroBlaze and remove the ILAs.

Writing the Software in SDK

Once the Vivado hardware has been generated the next step is to write application software which will configure the imager and the IP cores on the video processing core.

As such the software will perform the following

  • Initialize the AXI IIC, VTC and Interrupt controller
  • Set up the interrupt controller to generate AXI related interrupts – this include three interrupt service routines. One each for IIC transmit, receive and status.
  • Configure the timing on the VTC for the 10 inch display
  • Reset the camera over I2C and illuminate the lED on the PMOD
  • Detect the camera is present over I2C we are looking to detect a MT9M114
  • Initialize the camera over the I2C link – This will take a few seconds to write all of the commands

To initialize the imager, I have converted the Zynq based libraries as provided with the TDM114 example design into a format which can be used with the AXI IIC.

Once the camera has been initialized we will be able to see video on the ILA which is connected to the video stream to AXI Stream component.

Monitoring the I2C communication on the back of the TDNext Pmod shows the communication between the Arty S7 and the TDNext.

Once the camera has been detected, the application will download several I2C camera configuration settings.

Progress will be reported using the AXI UART

Once the camera is initialized we can use the ILA to verify, video is being generated by the imager and it is at the resolution we configured.

We do this by using a ILA and examining the video directly as it is received within the FPGA.

The image above shows a line width of 1280 pixel, which is exactly what we are expecting.

Received pixels are converted from parallel format into an AXI Stream.

AXI Stream is a unidirectional bus used to transfer data from a master to a slave, as a stream of data it does not contain an address channel. To control flow and communicate video timing information over the AXI Stream the following signals are used

  • TReady – Asserted bythe downstream peripheral when ready to receive data
  • TValid – Asserted bytransmitting peripheral when output data is valid
  • TUser – Issued for start of Frame
  • TLast – Issued for end of line

The second ILA can be used to ensure the AXI Stream is being correctly generated.

As we have no VDMA it is important the video output on the AXIS stream is a contiguus block and that TValid does not assert and deassert during the active pixel period.

We can ensure Tvalid is contiguous by using the pixel clock for the image processing chain.

The Library API used in this project are below, with the exception of camera_initial.h which contains IIC configuration data. All other header files are provided by Xilinx based upon the hardware configuration.

#include 
#include "platform.h"
#include "xiic.h"
#include "xintc.h"
#include "xil_exception.h"
#include "camera_initial.h"
#include "xvtc.h"
#include "xv_demosaic.h"

Devices addresses and identifiers

#define IIC_dev 			XPAR_IIC_0_DEVICE_ID
#define int_dev 			XPAR_INTC_0_DEVICE_ID
#define IIC_SLAVE_ADDR		        0x48
#define PCA9534_IIC_ADDR                0x20
#define INTC_DEVICE_INT_ID	        XPAR_INTC_0_IIC_0_VEC_ID
#define BUFFER_SIZE		        6

The main loop of the application can be seen below

int main()
{
   u32  Status;
   XIic_Config *iic_conf;
   XVtc	VtcInst;
   XVtc_Config *vtc_config;
   XVtc_Timing vtcTiming;
   XVtc_SourceSelect SourceSelect;
   XV_demosaic_Config *mosaic_config;
   init_platform();
   printf("www.adiuvoengineering.com S7 Imager example\n\r");
   mosaic_config = XV_demosaic_LookupConfig(XPAR_XV_DEMOSAIC_0_DEVICE_ID);
   XV_demosaic_CfgInitialize(&mosaic,mosaic_config,mosaic_config->BaseAddress);
   XIntc_Initialize(&InterruptController, int_dev);
   SetUpInterruptSystem();
   iic_conf = XIic_LookupConfig(IIC_dev);
   Status =  XIic_CfgInitialize(&iic, iic_conf, iic_conf->BaseAddress);
   if (Status != XST_SUCCESS) {
	 printf("XIic initial is fail \n \r") ;
	return XST_FAILURE;
   }
   XIic_SetSendHandler(&iic, &iic,	(XIic_Handler) SendHandler);
   XIic_SetRecvHandler(&iic, &iic, (XIic_Handler) ReceiveHandler);
   XIic_SetStatusHandler(&iic, &iic,(XIic_StatusHandler) StatusHandler);
   vtc_config = XVtc_LookupConfig(XPAR_VTC_0_DEVICE_ID);
   XVtc_CfgInitialize(&VtcInst, vtc_config, vtc_config->BaseAddress);
   vtcTiming.HActiveVideo = 1280;
   vtcTiming.HFrontPorch = 65;
   vtcTiming.HSyncWidth = 55;
   vtcTiming.HBackPorch = 40;
   vtcTiming.HSyncPolarity = 0;
   vtcTiming.VActiveVideo = 800;
   vtcTiming.V0FrontPorch = 7;//8;
   vtcTiming.V0SyncWidth = 4;
   vtcTiming.V0BackPorch = 12;
   vtcTiming.V1FrontPorch = 7;
   vtcTiming.V1SyncWidth = 4;
   vtcTiming.V1BackPorch = 12;
   vtcTiming.VSyncPolarity = 0;
   vtcTiming.Interlaced = 0;
   memset((void *)&SourceSelect, 0, sizeof(SourceSelect));
   SourceSelect.VBlankPolSrc = 1;
   SourceSelect.VSyncPolSrc = 1;
   SourceSelect.HBlankPolSrc = 1;
   SourceSelect.HSyncPolSrc = 1;
   SourceSelect.ActiveVideoPolSrc = 1;
   SourceSelect.ActiveChromaPolSrc= 1;
   SourceSelect.VChromaSrc = 1;
   SourceSelect.VActiveSrc = 1;
   SourceSelect.VBackPorchSrc = 1;
   SourceSelect.VSyncSrc = 1;
   SourceSelect.VFrontPorchSrc = 1;
   SourceSelect.VTotalSrc = 1;
   SourceSelect.HActiveSrc = 1;
   SourceSelect.HBackPorchSrc = 1;
   SourceSelect.HSyncSrc = 1;
   SourceSelect.HFrontPorchSrc = 1;
   SourceSelect.HTotalSrc = 1;
   XVtc_RegUpdateEnable(&VtcInst);
   XVtc_SetGeneratorTiming(&VtcInst,&vtcTiming);
   XVtc_SetSource(&VtcInst, &SourceSelect);
   XVtc_EnableGenerator(&VtcInst);
   XIic_Reset(&iic);
   PCA9534_CTRL ();
   Detect_Camera();
   Soft_Reset_Camera();
   Initial_Camera();
   
   XV_demosaic_Set_HwReg_width(&mosaic,0x500);
   XV_demosaic_Set_HwReg_height(&mosaic,0x31f);
   XV_demosaic_Set_HwReg_bayer_phase(&mosaic,0x1);
   XV_demosaic_EnableAutoRestart(&mosaic);
   XV_demosaic_Start(&mosaic);
   while(1){
   }
   cleanup_platform();
   return 0;
} 

Running the Entire software application resulted in me capturing the image below of the my conference badge collection.

I need to adjust some of the settings to provide to increase the integration time however, the basic image processing pipeline is working as we would expect.

Conclusion

It is easy to create a vision processing system which works directly with the imager in place of a camera. This often allows for a more cost efficient and potentially a more responsive solution as processing chain is significantly reduced.

Of course, it is also possible to include in HLS image processing blocks within the image processing chain if desired as we have done previously.

The post Creating an FPGA-Based Low-Cost Imaging System appeared first on SummerSolderS.



This post first appeared on Summersolders, please read the originial post: here

Share the post

Creating an FPGA-Based Low-Cost Imaging System

×

Subscribe to Summersolders

Get updates delivered right to your inbox!

Thank you for your subscription

×