Canadian Beef Cattle Survey

Production Lot Upload Data Preparation: Alberta Health Solutions (ABHS)
Site Information Details

Alberta Beef Health Solutions (ABHS)
Vet: CSBV00003 (3)
Project Year: 2021
The site has three (3) herds. They are:
   CSBH000027
   CSBH000028
   CSBH000029

Access to previous years documentation click on the appropriate link below:
   ABHS 2019
   ABHS 2020

Production Lot Upload Data Preparation: Alberta Health Solutions (ABHS)
Data Source Information

ABHS has 3 herds.
Each spreadsheet/workbook from ABHS represents a specific herd for a specific sampling year.
Within the workbook there is usually just 2 drugs: feed and injectable (parenteral).
The core fields in a sheet have the drug fields at the end of the core field data.
The drug fields are assembled as columns in the spreadsheet.

There are options how to setup each drug.
Each drug sheet can have all associated Core fields and Drug fields together.
Another option is to place all Drug field information, Feed and Injectable, at the end of the Core fields.
Depending on what option is used, the setup of the upload templates will vary.
Before creating and setting up a template, go thru all spreadsheets provided from the site.
Collect field names from each site and put them together. Review the field names to make sure they all
match. My experience is the field names do not match. To reduce the number of templates to create,
get the field names to match. The fields do not need to be in the same order.

The other reason to check all spreadsheets before creating a template is to see if there are the same number of drugs.
I have discovered it is the case the # of columns of drugs are not the same or there are more drugs in
a spreadsheet compared to another. To make it easier, prepare only 1 template, get all
drugs together across all drug routes for all spreadsheet data.
If you do it in this way, it is a possible route to introduce more errors. Be careful!

Production Lot Upload Data Preparation: Alberta Health Solutions (ABHS)
Data Source Information

For project year 2020 it is decided to provide more documentation how to prepare for ABHS data upload.
Here are some important notes to consider.

***** Please be reminded *****, ABHS 2021 source changed format from data coming in by columns of data to
the same or very similar format to VAHS 2021 data which has core data by columns and the drug
data provided in a "by row" format. These changes will be reflected in the template information below.

Note 0:
Create worksheet to work on feed called Work-Feed
Create worksheet to work on parenteral called Work-Injectable
Copy data over and format appropriately

Note 2:
Added column Closeout.
We are missing closeout category in the source data. It sometimes known as season.
Because of this we will calculate closeout from date in or date out (tba which is correct)

Note 3:
In Work-Feed worksheet for drug columns we added primary and
secondary reasons for use at the end of the drug column name
Primary reason codes:
   GP-Growth promotion
   DP-Disease Prevention
   DT-Disease Treatment

Secondary reasons added from original worksheet called feed.
These regimen information are in red below the feed source original data.

Note 4:
Combined the injectable and feed data in worksheet called:
Work-Injectable & Feed
We made sure the lot numbers are exactly the same.

Note 5:
Create a new worksheet
Call the new worksheet: To Upload-Herd 27 Feed & Inj
Copy and paste all the data into this worksheet from
Work-Injectable & Feed and paste into the new worksheet.

Herd 27 should be ready to upload.

Note 6: Preparing for first upload of raw data into the database
Open the database application
Connect to Development server
Check if Herd 27 for 2020 is created in the survey.
If not then add it.

If you attempt to add a new survey for Herd 27 for year 2020 and no herds are listed,
then you have to add allocated herd 27 for 2020

If you attempt to add a new allocation for Herd 27 for year 2020 and no herds are listed,
then you have to go to vet and herd management, find Herd 27 and add project 2020

Note 7:
Drug name fields should be changed.
For instance this drug name: tilmicosin-Treatment-Respiratory
Will be changed to: tilmicosin Treatment Respiratory
or to: tilmicosin DT Respiratory
The main thing is a "-" gets converted to null and a "blank" gets converted to _
Please take note, because it affects the field names in the db uploaded table.

Note 8:
Some column names in the source spreadsheet are reserved words for PostGreSQL field names.
Here is the list of names I change so far: (I will expand the list when the need arises)
   in -> n_in
   out -> n_out
   type -> type_uk

Note 9 December 13, 2021: Missing # of animals died
Add column into the spreadsheet or calculate using a transform in the template.
Adding transform for ABHS won't work because we uploaded from Source.
If template was populated from target then a transform could be done in the template.
We added 2 columns:
   # Died -> calculated from # HD - # Head Out
   # Railed -> all values set to 0

Note 10 December 13, 2021: Missing active ingredient for injectable
In ABHS 2020 data we are missing these two drugs for injectables in lookup drug table:
   chlortetracycline Treatment Lameness
   chlortetracycline Treatment Other
In other words we do not have Chlortetracycline in the active ingredients for injectables.
I checked if all values are 0 and they are not. The drug needs to be added to the list or some other solution.

Note 10A December 14, 2021: Missing active ingredient for injectable
I have confirmed with Dr. Sheryl Gow chlortetracycline is probably an Oxytetracycline 200 ml
But she does not know this. We need to contact vet and confirm.

Also, I had oxytetracycline injectables to upload but did not know what concentration levels.
Dr. Sheryl Gow has confirmed through regimen information is Oxy Tet 200
Update template accordingly.

Check with Dr. Sheryl Gow for source active ingredient field name
   ceftiofur CFA Treatment Lamenessn - code as Ceftiorfor 200
   ceftiofur NA Treatment Enteric - code as Ceftiorfor 50
Please check and check herd 28 and herd 29 if similar problem exists.

Note 11 December 13, 2021: Missing total # of head days
Add column into the spreadsheet for total # of head days: total days
   It is calculated by: =m2-k2
Make sure to format the column as a number then it won't attempt to return a peculiar date
Note 12 December 14, 2021: Future Changes
The template field table has fields to describe how data is transferrred from source to target.
I have two fields I would like to the template field table. Here they are:
   1. template field setup is complete
   2. template field is required
It be the template field is required should be added to the target field information. We will change it when there is time.

Production Lot Upload Data Preparation: Alberta Health Solutions (ABHS)
Template Setup Suggestions

A single upload Template can be setup for one or more years batch of data.
For the current document, assume we place all drug known fields for all drug routes at the end of the core fields.
When do this, then one Template is created.
The Template is populated from a worksheet from a specific spreadsheet or workbook.
The reason for this is because the drug fields are in columns not rows.
The system is designed to deal with it but you must upload by Source data to get it to work correctly.
The Template should be setup to have multiple or mix of drugs, data is by column and populated by source.

Here are the typical template settings used for ABHS
Name: ABHS 2021 - Core (73)
Level: Top Level (1)
Type: Core (1)
Route: Multiple Drugs (4)
Version: Version 1 (1)
By Row or Column: By Column (2)
Method: Populate template fields from Target


Core Template: Associated Core Template Fields

column_number target_column_classify_field_setup target_field_category target_column_type target_column_name source_column_name column_not_used
1 Setup Standard Field (1) Core Field (0) bigint csb_survey_production_lot_id csb_survey_production_lot_id Not Used (1)
2 Setup Standard Field (1) Core Field (0) bigint csb_survey_main_id cipars_feedlot_code Not Used (1)
3 Setup Standard Field (1) Core Field (0) bigint csb_record_group_id csb_record_group_id Not Used (1)
4 Setup Standard Field (1) Core Field (0) bigint csb_account_id csb_account_id Not Used (1)
5 Setup Standard Field (1) Core Field (0) bigint csb_survey_production_lot_upload_id csb_survey_production_lot_upload_id Not Used (1)
6 Setup Standard Field (1) Core Field (0) integer csb_pl_template_id csb_pl_template_id Not Used (1)
7 Setup Standard Field (1) Core Field (0) varchar(128) csb_pl_production_lot_code feedlot_lot_name Used (0)
8 Setup Lookup Field (2) Core Field (0) integer csb_pl_sex_id sex Used (0)
9 Setup Standard Field (1) Core Field (0) numeric csb_pl_sex_percent_steers csb_pl_sex_percent_steers Not Used (1)
10 Setup Standard Field (1) Core Field (0) numeric csb_pl_sex_percent_cows csb_pl_sex_percent_cows Not Used (1)
11 Setup Standard Field (1) Core Field (0) numeric csb_pl_sex_percent_heifers csb_pl_sex_percent_heifers Not Used (1)
12 Setup Standard Field (1) Core Field (0) numeric csb_pl_sex_percent_bulls csb_pl_sex_percent_bulls Not Used (1)
13 Setup Lookup Field (2) Core Field (0) integer csb_pl_animal_age_category_id type_uk Used (0)
14 Setup Standard Field (1) Core Field (0) numeric csb_pl_percent_calves csb_pl_percent_calves Not Used (1)
15 Setup Standard Field (1) Core Field (0) numeric csb_pl_percent_yearlings csb_pl_percent_yearlings Not Used (1)
16 Setup Standard Field (1) Core Field (0) numeric csb_pl_percent_adults csb_pl_percent_adults Not Used (1)
17 Setup Lookup Field (2) Core Field (0) integer csb_pl_origin_id origin Used (0)
18 Setup Standard Field (1) Core Field (0) numeric csb_pl_origin_percent_auction csb_pl_origin_percent_auction Not Used (1)
19 Setup Standard Field (1) Core Field (0) numeric csb_pl_origin_percent_backgrounded csb_pl_origin_percent_backgrounded Not Used (1)
20 Setup Standard Field (1) Core Field (0) numeric csb_pl_origin_percent_ranch_direct csb_pl_origin_percent_ranch_direct Not Used (1)
21 Setup Lookup Field (2) Core Field (0) integer csb_pl_brd_risk_id brd_risk_category Used (0)
22 Setup Standard Field (1) Core Field (0) numeric csb_pl_brd_risk_percent_low csb_pl_brd_risk_percent_low Not Used (1)
23 Setup Standard Field (1) Core Field (0) numeric csb_pl_brd_risk_percent_medium csb_pl_brd_risk_percent_medium Not Used (1)
24 Setup Standard Field (1) Core Field (0) numeric csb_pl_brd_risk_percent_high csb_pl_brd_risk_percent_high Not Used (1)
25 Setup Lookup Field (2) Core Field (0) integer csb_pl_quarter_of_year_closeout_id closeout Used (0)
26 Setup Standard Field (1) Core Field (0) integer csb_pl_n_animals_in hd_in Used (0)
27 Setup Standard Field (1) Core Field (0) integer csb_pl_n_animals_railed csb_pl_n_animals_railed Not Used (1)
28 Setup Standard Field (1) Core Field (0) integer csb_pl_n_animals_dead deaths Used (0)
29 Setup Standard Field (1) Core Field (0) integer csb_pl_n_animals_slaughtered hd_out Used (0)
30 Setup Lookup Field (2) Core Field (0) integer csb_pl_source_weight_in_units_id csb_pl_source_weight_in_units_id Not Used (1)
31 Setup Standard Field (1) Core Field (0) numeric csb_pl_source_weight_in_value csb_pl_source_weight_in_value Not Used (1)
32 Setup Multiple Target Fields (3) Core Field (0) numeric csb_pl_avg_weight_in_kgs avg_in_weight Used (0)
33 Setup Lookup Field (2) Core Field (0) integer csb_pl_source_weight_out_units_id csb_pl_source_weight_out_units_id Not Used (1)
34 Setup Standard Field (1) Core Field (0) numeric csb_pl_source_weight_out_value csb_pl_source_weight_out_value Not Used (1)
35 Setup Multiple Target Fields (3) Core Field (0) numeric csb_pl_avg_weight_out_kgs avg_out_weight Used (0)
36 Setup Standard Field (1) Core Field (0) date csb_pl_avg_date_in avg_in_date Used (0)
37 Setup Standard Field (1) Core Field (0) date csb_pl_avg_date_out avg_out_date Used (0)
38 Setup Standard Field (1) Core Field (0) numeric csb_pl_total_days_on_feed dof Used (0)
39 Setup Standard Field (1) Core Field (0) numeric csb_pl_total_head_days head_days Used (0)
40 Setup Standard Field (1) Core Field (0) varchar csb_pl_notes csb_pl_notes Not Used (1)





Name: ABHS 2021 - Feed Drugs (75)
Level: Second Level (2)
Type: Drug (2)
Route: Feed (2)
Version: Version 1 (1)
By Row or Column: By Row (1)
Method: Populate template fields from Target


Drug Template: Feed Drug Template Fields

column_number target_column_classify_field_setup target_field_category target_column_type target_column_name source_column_name column_not_used
1 Setup Standard Field (1) Core Field (0) bigint csb_survey_production_lot_drug_id lot Not Used (1)
2 Setup Standard Field (1) Core Field (0) bigint csb_survey_production_lot_id csb_survey_production_lot_id Not Used (1)
3 Setup Standard Field (1) Core Field (0) bigint csb_survey_main_id csb_survey_main_id Not Used (1)
4 Setup Standard Field (1) Core Field (0) bigint csb_record_group_id csb_record_group_id Not Used (1)
5 Setup Standard Field (1) Core Field (0) bigint csb_account_id csb_account_id Not Used (1)
6 Setup Standard Field (1) Core Field (0) bigint csb_survey_production_lot_upload_id csb_survey_production_lot_upload_id Not Used (1)
7 Setup Standard Field (1) Core Field (0) integer csb_pl_template_id csb_pl_template_id Not Used (1)
8 Setup Standard Field (1) Core Field (0) integer csb_pl_drug_ordinal csb_pl_drug_ordinal Not Used (1)
9 Setup Lookup Field (2) Core Field (0) integer csb_pl_drug_route_id route Used (0)
10 Setup Standard Field (1) Core Field (0) varchar(192) csb_pl_source_drug_name active_ingredient Used (0)
11 Setup Standard Field (1) Core Field (0) numeric csb_pl_source_drug_value total_mg Used (0)
12 Setup Lookup Field (2) Core Field (0) integer csb_pl_source_drug_value_units_id unit Used (0)
13 Setup Standard Field (1) Trade Name (2) integer csb_pl_drug_trade_name_id (TnId=) csb_pl_drug_trade_name_id Not Used (1)
14 Setup Standard Field (1) Active Ingredient (1) integer csb_pl_drug_active_ingredient_id (AiId=) active_ingredient Used (0)
15 Setup Standard Field (1) Core Field (0) numeric csb_pl_drug_total_mg total_mg Used (0)
16 Setup Standard Field (1) Core Field (0) numeric csb_pl_drug_total_ml csb_pl_drug_total_ml Not Used (1)
17 Setup Standard Field (1) Core Field (0) bigint csb_pl_drug_regimen_id csb_pl_drug_regimen_id Not Used (1)
18 Setup Standard Field (1) Core Field (0) bigint csb_pl_drug_regimen_detail_id csb_pl_drug_regimen_detail_id Not Used (1)
19 Setup Lookup Field (2) Core Field (0) integer csb_pl_drug_primary_reason_for_use tm Used (0)
20 Setup Standard Field (1) Core Field (0) numeric csb_pl_drug_primary_reason_for_use_pct_gp csb_pl_drug_primary_reason_for_use_pct_gp Not Used (1)
21 Setup Standard Field (1) Core Field (0) numeric csb_pl_drug_primary_reason_for_use_pct_dp csb_pl_drug_primary_reason_for_use_pct_dp Not Used (1)
22 Setup Standard Field (1) Core Field (0) numeric csb_pl_drug_primary_reason_for_use_pct_dt csb_pl_drug_primary_reason_for_use_pct_dt Not Used (1)
23 Setup Standard Field (1) Core Field (0) integer csb_pl_drug_secondary_reason_respiratory primary_use_reason Used (0)
24 Setup Standard Field (1) Core Field (0) integer csb_pl_drug_secondary_reason_digestive csb_pl_drug_secondary_reason_digestive Not Used (1)
25 Setup Standard Field (1) Core Field (0) integer csb_pl_drug_secondary_reason_lameness csb_pl_drug_secondary_reason_lameness Not Used (1)
26 Setup Standard Field (1) Core Field (0) integer csb_pl_drug_secondary_reason_neurologic csb_pl_drug_secondary_reason_neurologic Not Used (1)
27 Setup Standard Field (1) Core Field (0) integer csb_pl_drug_secondary_reason_liver_abscess csb_pl_drug_secondary_reason_liver_abscess Not Used (1)
28 Setup Standard Field (1) Core Field (0) integer csb_pl_drug_secondary_reason_coccidiosis csb_pl_drug_secondary_reason_coccidiosis Not Used (1)
29 Setup Standard Field (1) Core Field (0) integer csb_pl_drug_secondary_reason_enteritis csb_pl_drug_secondary_reason_enteritis Not Used (1)
30 Setup Standard Field (1) Core Field (0) integer csb_pl_drug_secondary_reason_mixed csb_pl_drug_secondary_reason_mixed Not Used (1)
31 Setup Standard Field (1) Core Field (0) varchar(128) csb_pl_drug_secondary_reason_mixed_description csb_pl_drug_secondary_reason_mixed_description Not Used (1)
32 Setup Standard Field (1) Core Field (0) integer csb_pl_drug_secondary_reason_other csb_pl_drug_secondary_reason_other Not Used (1)
33 Setup Standard Field (1) Core Field (0) varchar(128) csb_pl_drug_secondary_reason_other_description csb_pl_drug_secondary_reason_other_description Not Used (1)
34 Setup Standard Field (1) Core Field (0) varchar csb_pl_drug_secondary_reason_source primary_use_reason Used (0)
35 Setup Standard Field (1) Core Field (0) varchar csb_pl_drug_notes csb_pl_drug_notes Not Used (1)


Name: ABHS 2021 - Injectable Drugs (74)
Level: Second Level (2)
Type: Drug (2)
Route: Injectable (3)
Version: Version 1 (1)
By Row or Column: By Row (1)
Method: Populate template fields from Target


Drug Template: Injectable Drug Template Fields

column_number target_column_classify_field_setup target_field_category target_column_type target_column_name source_column_name column_not_used
1 Setup Standard Field (1) Core Field (0) bigint csb_survey_production_lot_drug_id lot Not Used (1)
2 Setup Standard Field (1) Core Field (0) bigint csb_survey_production_lot_id csb_survey_production_lot_id Not Used (1)
3 Setup Standard Field (1) Core Field (0) bigint csb_survey_main_id csb_survey_main_id Not Used (1)
4 Setup Standard Field (1) Core Field (0) bigint csb_record_group_id csb_record_group_id Not Used (1)
5 Setup Standard Field (1) Core Field (0) bigint csb_account_id csb_account_id Not Used (1)
6 Setup Standard Field (1) Core Field (0) bigint csb_survey_production_lot_upload_id csb_survey_production_lot_upload_id Not Used (1)
7 Setup Standard Field (1) Core Field (0) integer csb_pl_template_id csb_pl_template_id Not Used (1)
8 Setup Standard Field (1) Core Field (0) integer csb_pl_drug_ordinal csb_pl_drug_ordinal Not Used (1)
9 Setup Lookup Field (2) Core Field (0) integer csb_pl_drug_route_id route Used (0)
10 Setup Standard Field (1) Core Field (0) varchar(192) csb_pl_source_drug_name active_ingredient Used (0)
11 Setup Standard Field (1) Core Field (0) numeric csb_pl_source_drug_value sum_of_total_mg Used (0)
12 Setup Lookup Field (2) Core Field (0) integer csb_pl_source_drug_value_units_id unit Used (0)
13 Setup Standard Field (1) Trade Name (2) integer csb_pl_drug_trade_name_id (TnId=) csb_pl_drug_trade_name_id Not Used (1)
14 Setup Standard Field (1) Active Ingredient (1) integer csb_pl_drug_active_ingredient_id (AiId=) active_ingredient Used (0)
15 Setup Standard Field (1) Core Field (0) numeric csb_pl_drug_total_mg sum_of_total_mg Used (0)
16 Setup Standard Field (1) Core Field (0) numeric csb_pl_drug_total_ml csb_pl_drug_total_ml Not Used (1)
17 Setup Standard Field (1) Core Field (0) bigint csb_pl_drug_regimen_id csb_pl_drug_regimen_id Not Used (1)
18 Setup Standard Field (1) Core Field (0) bigint csb_pl_drug_regimen_detail_id csb_pl_drug_regimen_detail_id Not Used (1)
19 Setup Lookup Field (2) Core Field (0) integer csb_pl_drug_primary_reason_for_use tm Used (0)
20 Setup Standard Field (1) Core Field (0) numeric csb_pl_drug_primary_reason_for_use_pct_gp csb_pl_drug_primary_reason_for_use_pct_gp Not Used (1)
21 Setup Standard Field (1) Core Field (0) numeric csb_pl_drug_primary_reason_for_use_pct_dp csb_pl_drug_primary_reason_for_use_pct_dp Not Used (1)
22 Setup Standard Field (1) Core Field (0) numeric csb_pl_drug_primary_reason_for_use_pct_dt csb_pl_drug_primary_reason_for_use_pct_dt Not Used (1)
23 Setup Standard Field (1) Core Field (0) integer csb_pl_drug_secondary_reason_respiratory use_reason Used (0)
24 Setup Standard Field (1) Core Field (0) integer csb_pl_drug_secondary_reason_digestive csb_pl_drug_secondary_reason_digestive Not Used (1)
25 Setup Standard Field (1) Core Field (0) integer csb_pl_drug_secondary_reason_lameness csb_pl_drug_secondary_reason_lameness Not Used (1)
26 Setup Standard Field (1) Core Field (0) integer csb_pl_drug_secondary_reason_neurologic csb_pl_drug_secondary_reason_neurologic Not Used (1)
27 Setup Standard Field (1) Core Field (0) integer csb_pl_drug_secondary_reason_liver_abscess csb_pl_drug_secondary_reason_liver_abscess Not Used (1)
28 Setup Standard Field (1) Core Field (0) integer csb_pl_drug_secondary_reason_coccidiosis csb_pl_drug_secondary_reason_coccidiosis Not Used (1)
29 Setup Standard Field (1) Core Field (0) integer csb_pl_drug_secondary_reason_enteritis csb_pl_drug_secondary_reason_enteritis Not Used (1)
30 Setup Standard Field (1) Core Field (0) integer csb_pl_drug_secondary_reason_mixed csb_pl_drug_secondary_reason_mixed Not Used (1)
31 Setup Standard Field (1) Core Field (0) varchar(128) csb_pl_drug_secondary_reason_mixed_description csb_pl_drug_secondary_reason_mixed_description Not Used (1)
32 Setup Standard Field (1) Core Field (0) integer csb_pl_drug_secondary_reason_other csb_pl_drug_secondary_reason_other Not Used (1)
33 Setup Standard Field (1) Core Field (0) varchar(128) csb_pl_drug_secondary_reason_other_description csb_pl_drug_secondary_reason_other_description Not Used (1)
34 Setup Standard Field (1) Core Field (0) varchar csb_pl_drug_secondary_reason_source use_reason Used (0)
35 Setup Standard Field (1) Core Field (0) varchar csb_pl_drug_notes csb_pl_drug_notes Not Used (1)