APPENDIX 1: EVALUATION METHOD

The project used a mixed-methods approach combining quantitative and qualitative research methods.

Quantitative method

Phase 1 - Initial survey

The initial survey was a short 3-5 minute phone screen of employers in October - November 2009 to identify knowledge of and the prevalence of trial periods, rest and meal breaks and breastfeeding breaks and facilities among employers. Employers were also asked if they were willing to take part in the follow-up survey. Consenting employers were then sent an online link to this follow-up survey, or were posted a paper copy, based on their choice. Only the findings of trial periods are included in this report.

Sampling frame

The sampling frame for the initial survey was a Department of Labour employer database. When incomplete, duplicate, and non-employing entries were removed, the database produced a sampling frame of 33,576 employers of which a random sample of 3,600 employers was drawn. Although the sample was not a random probability sample of New Zealand businesses, it was an approximate match of the actual industry distribution in New Zealand.

Response rate

The sample size for the employer survey was intended to be 1200, across all employer sizes. Only the small employers (1-19 employees) completed the section on trial periods.

Of the 3,532 employers contacted:

• 1,391 (39%) completed the initial survey, of whom 771 went on to do the follow-up survey (10% by post, and 90% online)
• 324 (9%) declined to take part
• 685 (19%) were unattainable because of invalid contact details
• 1,132 (32%) were unavailable

Business size distribution

The following table shows the business size distribution of survey respondents, compared to the population.

Table 14: Business size distribution of the initial survey compared to the population

Size of business	Number in initial survey	Percent in initial survey	Percent in population[10]
1-19 employees (small)	989	71	91
20-49 employees (medium)	206	15	6
50+ employees (large)	196	14	3
Total	1391	100	100

Industry distribution

Table 15: Industry distribution of the initial survey compared to the population

Industry	Number in initial survey	Percent in initial survey	Percent in population[11]
A Agriculture, Forestry and Fishing	59	4	14
B Mining	4	0	0
C Manufacturing	207	15	8
D Electricity, Gas, Water and Waste	1	0	0
E Construction	109	8	13
F Wholesale Trade	105	8	6
G Retail Trade	233	17	9
H Accommodation and Food Services	88	6	7
I Transport, Postal and Warehousing	71	5	3
J Information Media and Telecommunications	5	0	1
K Financial and Insurance Services	27	2	2
L Rental, Hiring and Real Estate	210	15	4
M Professional, Scientific and Technical	31	2	10
N Administrative and Support Services	42	3	3
O Public Administration and Safety	97	7	0
P Education and Training	28	2	4
Q Health Care and Social Assistance	42	3	5
R Arts and Recreation Services	10	1	2
S Other Services	2	0	8
Unknown	20	1
Total	1391	100	100

Phase 2 - Follow-up Survey

The follow-up survey was 8-14 minutes duration, and administered over the internet or by postal questionnaire, depending on the employer's preference. The aim of the survey was to obtain a more comprehensive understanding of employers' knowledge of the amendments, any changes employers had made, and their perceptions of compliance costs relative to benefits. To maintain respondent confidentiality, responses from the initial and follow-up surveys could not be linked, thus some questions from the initial survey were repeated in the follow-up survey.

Consenting employers from the initial survey made up the sampling frame for the follow-up survey. The sample size across all three topics was intended to be between 400 and 500 employers.

The follow-up survey was completed online or on paper by 771 employers with a completion rate of approximately 88%. These respondents are a subset of the respondents from the initial survey. 989 small employers (1-19 employees) took part in the initial survey, of which 527 small employers went on to complete the follow-up survey.

Respondents self-reported their business size, industry and location.

Paper survey responses were entered into the online survey tool. These results were then imported into SPSS, which was used to analyst the results. Descriptive statistics are reported here. A t-test was used to compare the difference between 2 proportions for cells with at least 40 respondents. Only results which were significantly different at the 95% confidence interval are reported here.

Business size distribution

Table 16: Business size distribution of the follow-up survey compared to the population

Size of business	Number in follow-up survey	Percent in follow-up survey	Percent in population[12]
1-19 employees (small)	527	69	91
20-49 employees (medium)	117	15	6
50+ employees (large)	127	17	3
Total	771	100	100

Industry distribution

Respondents in the follow-up survey self-reported their industry. However, nearly 19% selected 'Other Services' and described their business. These businesses were allocated appropriate industry codes based on their description.

There were no significant differences by industry for any of the trial period questions in either survey. This result may be partly explained by small base numbers for some questions.

Table 17: Small firm (1-19 employees) industry distribution of the follow-up survey compared to the population (n=527)

Industry	Number	Percent	Percent in population[13]
A Agriculture, Forestry and Fishing	32	6	15
B Mining	1	0	0
C Manufacturing	53	10	7
D Electricity, Gas, Water & Waste Services	9	2	0
E Construction	46	9	13
F Wholesale Trade	32	6	6
G Retail Trade	100	19	10
H Accommodation and Food Services	27	5	7
I Transport, Postal and Warehousing	30	6	3
J Information Media and Telecommunications	7	1	1
K Financial and Insurance Services	27	5	2
L Rental, Hiring & Real Estate Services	6	1	4
M Professional, Scientific and Technical Services	53	10	10
N Administrative and Support Services	10	2	3
O Public Administration and Safety	2	0	0
P Education and Training	24	5	3
Q Health care and Social Assistance	34	7	5
R Arts and Recreation Services	6	1	2
S Other Services	28	5	8
Total	527	100	100

Employer location

Table 18: Location of firms, follow-up survey (n=514)

Location	Number	Percent
Auckland	162	32
Auckland and North Island	3	1
Hamilton	21	4
Wellington	47	9
Christchurch	37	7
Canterbury	2	0
Dunedin	8	2
Main centres	5	1
North Island	168	33
South Island	61	12
Total	514	100

Qualitative method

The purpose of the qualitative research was to get a more in-depth understanding of the implementation and impact of trial periods, and the knowledge or awareness of employers and employees.

It consisted of semi-structured interviews with 31 people in total including 15 employers, 13 employees, 2 union officials, and 1 person with a general perspective. Some employees had been on more than 1 trial period, so could provide a variety of perspectives. Similarly, some employers had experience of more than one employee on a trial period.

Employers were recruited using the employer database mentioned above, the Yellow pages, employees recruited for this research, and through personal contacts. Employees were recruited through some employers recruited for this research, a newspaper advertisement, an advertisement at Student Job Search, the Council of Trade Unions and personal contacts. Union officials were recruited through the Council of Trade Unions as well.

The Wellington office of New Zealand Prisoners Aid and Rehabilitation Society (NZPARS) advised that prisoners had not got employment through trial periods, so NZPARS was not used to recruit respondents.

Most interviews were conducted face to face, and most were audio recorded with the interviewee's permission, or else extensive notes were hand written. All interviewees were offered a $50 supermarket voucher as gratuity. Respondents from throughout New Zealand were interviewed, however they were mainly from the Wellington region and Wairarapa.

Respondents were from the following industries:

Table 19: Industry distribution of qualitative research respondents^[14]

Industry	Number
A Agriculture, Forestry and Fishing	1
B Mining	0
C Manufacturing	3
D Electricity, Gas, Water & Waste Services	0
E Construction	3
F Wholesale Trade	0
G Retail Trade	3
H Accommodation and Food Services	7
I Transport, Postal and Warehousing	1
J Information Media and Telecommunications	1
K Financial and Insurance Services	0
L Rental, Hiring & Real Estate Services	0
M Professional, Scientific and Technical Services	1
N Administrative and Support Services	1
O Public Administration and Safety	1
P Education and Training	2
Q Health care and Social Assistance	0
R Arts and Recreation Services	2
S Other Services	6
Unknown	1
Total	33

Limitations

• Due to resource constraints, this evaluation included employees only in the qualitative, not quantitative research. As a result, it provides an understanding of issues for employees on trial periods, but does not give an idea of how widespread these issues are.
• The employer database, which was the sampling frame for the initial survey, had incomplete employer contact information. As a result, many employers had to be excluded from the sampling frame. Hence, the survey results may not be truly representative of employers in the country. However, we are confident that the overall results of the project are substantially indicative of employers because of the strong convergence between the qualitative and quantitative results.
• Trying to ascertain macro-economic impacts of and influences on trial periods, such as the wider labour market, was out of scope of this study.

 

---