Why the Need for the Package?
In the 1990s, concern over growing packer concentration and a hog industry market shock resulted in discontent among producers and packers. As a result, the US Congress passed the Livestock Mandatory Reporting Act of 1999 (1999 Act) [Pub. L. 106-78, Title IX] which is required to be reauthorized every 5 years. See here for a full history of the Livestock Mandatory Reporting Background.
Market reports were publicly issued in the form of .txt files with varying frequency from April 2000 to April 2020. Current and historical data were also housed in a USDA-AMS database that could also be accessed through the https://mpr.datamart.ams.usda.gov/ website. In an effort to make data more accessible the USDA-AMS began to develop APIs that could interface directly with this database. However, starting in April 2020, text files were made permanently unavailable leaving direct access at https://mpr.datamart.ams.usda.gov/ or API access the only plausible methods to access the data . Figure 1 displays the message which is now displayed when accessing text files.
The R programming language is a free software used for statistical computing and graphics that is widely used among statisticians and data analysts in industry academics, and government. It allows users to create packages/libraries that can be downloaded and distributed for free. Using the API documentation provided by USDA-AMS (see here for detailed API documentation), we developed an R package entitled ‘usdampr’ whose functions directly access the USDA-AMS databases. Its primary purpose is to allow greater public access to USDA-AMS data for those not familiar or comfortable directly accessing the data via API and is a viable alternative to now extinct text files and direct downloads from https://mpr.ams.usda.gov.
‘mpr_request’ is the primary function in the ‘usdampr’ package to request data from the United States Department of Agriculture - Agricultural Marketing Service (USDA-AMS) mandatory price reporting, commonly known as MPR. This function allows users to access data documented in the Livestock Mandatory Price Reporting (LMPR), Dairy Products Mandatory Reporting Program (DPMRP), and Federal Milk Marketing Orders (FMMOS) market reports. LMPR contains data for cattle, hogs, sheep, beef, pork, and lamb. The ‘usdampr’ package also contains a data set called ‘slugInfo’, which provides a reference dataframe for available reports and report identifiers to be used for making valid data requests through the ‘mpr_request’ function.
Functionality of Package
We now show some of the functionality of the ‘usdampr’ package. First, we demonstrate the flexibility in calling different reports and dates. Second, we demonstrate how downloaded results can be easily visualized with the aid of some graphics packages in R.> Available Reports to Pull
The fundamental inputs required to request data from USDA-AMS are a report identifier and date. The report identifier, commonly referred to as the slug id, can be supplied using new slug id’s from USDA-AMS or traditional or legacy slug id’s. For example, to pull the National Weekly Boxed Beef Cutout & Boxed Beef Cuts report one can either supply the slug ID (2461) or the legacy slug IDLM_XB459. For individuals who may not be familiar with the new slug id’s, legacy slug id’s, or report name we have provided a dataset called ‘slugInfo’ which displays this information for reference. Also included in ‘slugInfo’ is the commodity associated with each report, report frequency (e.g. daily, weekly, monthly, or annual) and report section headings. Figure 2is a snapshot of what the dataframe ‘slugInfo’ looks like.Data Request Examples
The data request requires two necessary inputs. The first input is slug ID or legacy slug ID. Slug ID should be a 4-digit number (numbers in characters are fine). Examples for slug ID include 2461 (Report name: National Weekly Boxed Beef Cutout & Boxed Beef Cuts), 2472 (Report name: Weekly Direct Slaughter Cattle). If you happen to not know the slug ID, you can use the legacy slug IDs, such as LM_XB459 (Report name: National Weekly Boxed Beef Cutout & Boxed Beef Cuts). When legacy slug IDs are provided, the mpr_request function performs an internal search for the corresponding slug IDs and then makes the data request. Users should provide either slug IDs or legacy slug IDs, not both. The provides slug IDs or legacy slug IDs must be valid. Use data(slugInfo) to get a list of valid slug IDs and the report information.
Figure 3displays the flexibility of the function ‘mpr_request’ using several different data requests. We display how the function accommodates a combination of multiple dates and multiple slug id’s. Likewise, we display data requests to data contained within LMPR, DPMPR, and FMMOS.
Data requested from https://mpr.datamart.ams.usda.gov/ is returned as a list with each sub-list containing different report sections. This allows each report section to be accessed and analyzed accordingly. This is done for two reasons. First, the format and structure of each report varies greatly both within and across commodities. The minimal benefit of requesting detailed section specific requests would come at the cost of highly complicating package development. Second, USDA-AMS currently limits the number of data requests per day. Excessive use / abuse of requests results in the user being blocked. Thus, to allow for a parsimonious package and to limit possibility of users being blocked access to data as a result of too many requests, we request all sections within a report and allow the user to specify specific report sections to use.Visualizing Data Requests
We know displays the ease by which requested data can be plotted using other frequently used packages. We use the 5 area weekly weighted average direct slaughter cattle premiums and discounts (LM_CT169 or 2489) from Jan 2018 to June 2020 as a working example. Figure 4 displays the code to pull the data and figure 5 displays the resulting graph. Requested data can be easily requested, processed/cleaned, using currently available packages in R.
This document has introduced the R package ‘usdampr’ which allows users to access the USDA-AMS mandatory price reporting database. It has documented the need for the package, relevant examples for accessing LMPR, DPMPR, and FMMOS data requiring only slug id’s and date range, and how the data can be displayed. Questions regarding package development or bugs can be directed to Elliott J. Dennis via email at email@example.com or 402-472-2164.