Catching up with the National Turfgrass Evaluation Program

We emailed some questions to Kevin Morris, executive director of the National Turfgrass Evaluation Program (NTEP), Beltsville, MD to update us on how NTEP is improving its trial evaluations and handling the resulting data:

Q: What qualifies a research program to be involved with NTEP?

Morris: NTEP has trials in approximately 35 US states and Canadian provinces; some states have more than one test site (e.g., California, Georgia and others). Every site that runs our “standard” trial evaluations is university-based (i.e., public institutions, most of which are land-grant university affiliated).

“Ancillary” trial sites evaluate a specific trait (such as a particular disease or stress, like traffic tolerance) and can be located off of a university research facility (most are on golf courses). We also conduct some trials strictly “on-site,” i.e., on an actual use site, such as a golf course.

An example of an on-site trial is our current fairway overseeding trial that is planted and evaluated on nine golf courses across the southern US. All NTEP test sites, whether standard, ancillary or on-site are coordinated by a university turfgrass researcher. Researchers are trained in turfgrass evaluation either by NTEP, or having an NTEP-trained researcher that instructs a graduate student or technical staff.

Q: What changes are in the works for NTEP’s trial evaluations? How much change in testing sites has there been over the years?

Morris: From the start of NTEP in 1980, there have been several significant changes in the way NTEP conducts its evaluations. Before 1992, sites conducted NTEP trials without any contract or agreement, and without compensation. In 1992, NTEP made the change to compensate its trial sites, but also instituted a grant-in-aid agreement dictating trial protocols and requiring each site to collect specific data. Since then, we instituted our training program to further standardize data collection. We now have a stable of excellent cooperators that invest significant time, funds and effort into make NTEP trials the best they can be.

Q: How has technology impacted how you collect data? And do you compile all the results as well as disseminate?

Morris: Many turfgrass evaluations, in particular turfgrass quality, are subjective ratings that are in effect, an opinion. Turfgrass quality ratings are a combination of color, texture, density, uniformity, disease damage, weeds and other factors that affect turf appearance and performance, all rolled into a 1-9 rating, 9=ideal turf. Turf quality ratings are collected monthly throughout the growing season and with 15-25 researchers collecting data for each trial, it is that summary of “opinions” that make turfgrass quality ratings useful.

However, much other data that either describes each entry, (such as color, texture and density) or documents performance due to a stress (such as drought, disease or traffic) is collected by NTEP. Some of this data can now be collected using digital image technology. Digital images are collected using a digital camera mounted on a 1-meter square metal box that is enclosed on all but one side. A light source is contained inside the box so that consistent light is available for each picture. The digital images are then analyzed by proprietary software for the percent green cover, and the range of green color is identified. This technology is very useful for ratings such as the actual green color (i.e., what shade of green is the entry?), or percent cover of the entry in a test plot.

Many NTEP researchers use this technology for specific data collection applications, as the results are very consistent. However, since a turf quality rating is subjective and an opinion, and a digital image measures percent green cover and shades of green, ratings of 1-9 are still used for turf quality and some other ratings.

In addition, if mixtures of grasses are present (as in some NTEP trials) or a disease or another stress is present (or multiple diseases or stresses), this complicates the digital image analysis. In the future, NTEP will investigate the use of more digital image technology, or other technology (such as flying a drone over an entire trial and collecting all images at one time). For now, though, most NTEP data will continue to consist of 1-9 scale ratings.

Yes, we compile all data and statistical analysis is performed on each data set, and any inconsistent data is removed. Then we publish data from each trial, each year on We are currently testing over 600 cultivars and experimental grasses of about 20 species, so this is a monumental task. In addition, at the end of each trial, we produce a 5-year summary report. In 2018, we have 12 annual reports and four final summary reports, which takes us much of the year to complete.

Q: Who are the most common users of NTEP data? How can it be accessed?

Morris: Users of NTEP data range from turfgrass breeders working to develop new varieties, to seed companies marketing new grasses, to professionals (golf course superintendents, athletic field managers, lawn care operators) trying to select the best grass for their situation, to specification developers (i.e., federal, state and local turf managers/golf course architects/athletic field builders, etc.) that must “spec” grass seed cultivars for bids, to brokers that purchase and sell grass seed worldwide based on NTEP data, to the homeowner. Therefore, our users are highly varied in location, application and expertise, and our data must serve that very broad clientele.

Data from back to the early 1980’s can be found on our website for free! This service exists for the turfgrass industry due primarily to entry fees paid by sponsoring companies, as well as the USDA-Agricultural Research Service in Beltsville, MD (USDA provides office space and other facilities for NTEP). Since our start in 1980, we have collected upwards of 1,000,000 data records on turfgrass. A data record is all the data collected for one year on a single 25 square foot plot at one location. We currently have more 35,000 plots across North America).

However, since our data is so broad and varied, we know that accessing it is not very ’user-friendly.’ Due to a USDA grant that NTEP is cooperating with, changes will be taking place in the future as a database of turfgrass evaluation data is being developed, as well as a system of menus and other methods to make NTEP data easier to access. This will make NTEP much more useful to the end user, including the athletic field manager.