Framework for Implementing Constructability Throughout Project Development from NEPA to Final Design (2023)

Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

202 CHAPTER 3. PHASE II: CRP DECISION-MAKING FRAMEWORK AND OTHER DELIVERABLES TASK 2-1: DECISION-MAKING FRAMEWORK The CRP Decision-Making Framework is the most important element of the research which involved creating a conceptual framework to Phase I of the research. The foundation of the final framework was the Conceptual Decision-Making Framework developed as part of Phase I (Task I-3). While the Conceptual Framework only answered the question of what level of CRP an agency should employ on a specific project, the final CRP Decision-Making Framework goes much further. The Framework Upon abandoning the style and scope of the Conceptual Framework, the team settled on four questions decision-makers would typically ask in the process of establishing CRPs for the projects in their work plans: 1) Does this project warrant a formalized CRP; and if so, how formal should it be? (Framework Output 1) 2) Which tools should be employed in the CRP for this project? (Framework Output 2) 3) Which people should be involved in the CRP on this project? (Framework Output 3) 4) When should the CRP commence, and at what milestones should the CRP be re- deployed? (Framework Output 4). To make a software tool that would generate the answers to these questions, the team established two forms of input: 1) the initial input data to establish a permanent model that is able, first, to receive user input and, then, to process that input to render the appropriate answers (outputs); and 2) the user’s project-specific input data, processed by the permanent model to formulate those outputs. Early in the development of this new framework, team members recognized that an electronic model would be essential to delivering a framework able to consider all user inputs and render all four outputs. A computer scientist was added to the team and, from that point on, the CRP Decision-Making Framework and the electronic model were developed in tandem.

203 The team had to decide on and establish the proper format of the original input—i.e., the data that would enable an algorithm to formulate the outputs. To this end, team members began work on a questionnaire, generating questions they considered germane to the four framework output questions. This process produced well over 40 questions, some of which were not pertinent to the outputs. Thus, the team decided to shorten the list of questions, to make the process more efficient and user-friendly. To develop this shorter list, two subtasks were executed simultaneously. One subtask was to analyze the questions to see if any were unnecessary or if any could be combined. The other was to tie each question to one, two, three, or all four outputs. At the end of this process, the model had 29 questions, each tied to its pertinent output(s). Table 3-1 shows this process for the first six questions. (See the entire table in Appendix C.) CRP Framework Input Questions Key to Outputs 1 – Should a formal CRP be performed on this project, and if so, to how formal? 2 – Which tools should be employed in the CRP for this project? 3 – Which people should be involved in the CRP on this project? 4 – When should the CRP commence, and at what milestones should the CRP be re- deployed?

204 Table 3-1. Tying Questions to Outputs in the Framework (Sample) The next subtask was to assign weights to each question, within each output. Some questions were more important than others within each output; also, certain questions were more important to certain outputs. Just as the formulating, grouping, and weighting of the questions had come together, the first version of the electronic model was ready to be tested. Therefore, the questions were each tied by coding to their pertinent output(s) in the electronic model; and, then, the weights the PI had calculated for each question were tied to the outputs, as appropriate. The weights within each output summed to 1.0000. When project data were fed into the model, it functioned properly, rendering all four outputs. Table 3-2 shows the weights for the first five questions for Output #1. Questions Outputs Project Description n/a 1. What is the project type? a. Paving b. Bridge c. Intersection d. New Const. e. Signalization f. RRR 1, 2, 3, 4 2. What is the estimated cost of this project? ($ millions) a. 0 - 15 b. 15+ - 50 c. 50+ - 100 d. 100+ - 500 e. > 500 1,4 3. Which delivery system will be used to build the project? a. D-B-B b. D-B c. CM/GC d. 3P e. IPD f. PD-B 1, 3, 4 4. Rate the political sensitivity of the project (5-highest; 1-lowest) 1 2 3 4 5 1, 2, 3, 4 5. Rate the environmental sensitivity of the project (5-highest; 1-lowest) 1 2 3 4 5 1, 2, 3, 4

205 CRP Framework Input Questions and Weights for Output 1 “Should a CRP be performed on this project, and if so, to what level?” Table 3-2. Weights for Questions 1-5 for Output #1. Questions Weights GENERAL 1. What is the project type? a. Paving b. Bridge c. Intersection d. New Const. e. Signalization f. RRR 0.0263 2. What delivery system will be used to build the project? a. D-B-B b. D-B c. CM/GC d. 3P e. IPD f. PD-B 0.0270 3. What is the estimated cost of this project? ($ millions) 1. 0 - 15 2. 15+ - 50 3. 50+ - 100 4. 100+ - 500 5. > 500 0.0182 4. Rate the political sensitivity of the project (5-highest; 1-lowest) 1 2 3 4 5 0.0263 5. Rate the environmental sensitivity of the project (5-highest; 1-lowest) 1 2 3 4 5 0.0293 While the team was encouraged by the model’s performance, this “alpha” version had two issues. First, the weights were the products of one person’s experience and knowledge, reflecting that person’s inherent biases. Second, the team was not sure that the weights were even related to the outputs—especially Outputs 2 and 3, regarding tools and people. After much discussion, each team member generated his or her own weights. These were integrated into the PI’s weighting, a process that rendered a better set of weights, since the experience, knowledge, and biases of one person were smoothed out by the inclusion of eight new sets of weights, with their inherent personal influences. Soon, the team ran data from many projects through the model, and the results were mixed. When the results were compared to the opinions of project personnel, many times the model’s recommendations made no sense especially where tools and people were concerned. Finally, the team arrived at the conclusion that the weights were useful in determining the following:

206 • Whether a CRP was necessary for a project. • How formal the process should be. • When the process should begin. • How often the CRP should be revisited. The weights were ineffective at recommending the proper tools and people for the CRP. Therefore, the team began to pursue a different approach to arriving at these types of recommendations. Each possible response to each question about Output 2 (tools) and Output 3 (people) would have to trigger a recommendation. For instance, for a question about the probability of geotechnical problems on a project, the possible responses were on a five-point Likert scale, where “5” represented the highest probability of problems, and “1” represented the lowest probability of problems. In this scenario, for Output 3, an answer of “1” or “2” would elicit no recommendation from the model. A response of “3” or “4” would cause the model to recommend that “a representative from the District Geotechnical Office, or lab, be involved in the CRP,” while an answer of “5” would cause the model to recommend that the District Geotechnical Engineer participate in person. In the end, the team did create and code just such a series of recommendations into the model. The same sort of logic was coded into the model for Output 2 regarding which tools to use in the CRP for that test project. Meanwhile, no one was satisfied with the weighting of the questions related to Output 1 (the appropriateness and intensity of a CRP on a project) or the questions related to Output 4 (when the CRP should commence, and how often the group should meet). First, the incorporation of eight people’s opinions on these weights was not sufficient for the necessary robustness of the tool; and second, the method of having people conjure up a number for each question, and then using interpolation to have all the weights sum up to 1.0000, was inadequate. The solution to the first problem was to simply involve more people. The solution to the second was to create or discover a means of generating the weights by asking dozens of people to participate in the

207 exercise. Both of these possible solutions prompted the team to rethink its vision of how to conduct the workshop component of the research methodology. Originally, the workshop (Task 2-2) for the research was to be used to introduce the framework to national leaders in the area of constructability for transportation projects. Then, the goal became to teach more people to use the framework—and, further, to urge them to champion its use in their home agencies and, thereby, foster the buy-in of agency leadership. Now, with the need to generate these weights, the team tapped into the experience of the workshop participants by using the AHP, an effective and widely accepted way to compare the importance of one thing against every other thing under consideration. First, the eight members of the team submitted to the AHP procedure. The Output 1 procedure, with 130 questions, took approximately 50 minutes. Since the exercise for Output 4 generated 210 questions, that procedure took approximately 90 minutes. The results were amalgamated into one weight for each question, and the weights all summed to 1.0000 for each output. At the workshop, more than 50 people (excluding the research team) sat for the AHP procedure, adding great robustness to the model. A copious amount of data was collected at the workshop. The model was introduced to the participants, and they were asked to break up into small breakout groups and use the model on two pre-packaged imaginary projects. Groups that finished the two projects quickly were allowed to run data from one of their own past projects through the model. The breakout groups then shared their impressions with the whole group. Each smaller group then wrote down and submitted suggestions to improve the model. After the workshop, the research team organized the data and analyzed each suggestion. Of the approximately 70 specific suggestions, many met with the approval of the team and were implemented. The Electronic Framework Model The RFP did not call for an electronic component to the CRP Decision-Making Framework, and the research team initially did not intend to produce one. Phase I ended with the team meeting face-to-face with the NCHRP 10-99 panel in November, 2019. At that meeting, the panel said

208 that they did not think that the conceptual model developed by the team in Phase I offered enough guidance to help agency leaders make the decisions necessary for an effective CRP. The conceptual model did a good job of recommending whether a particular project warranted a CRP, and how formal a process it should be. It also did a fair job of recommending how often CRP reviewers should re-visit the CR on a project. Still, the panel also wanted to know who should be involved in the CRP, which tools are best for the process, and when it should commence on a particular project. Once the team began to pursue a framework that could offer recommendations on this widened scope of issues, the need for an electronic model became apparent. The team could scarcely handle the volume of data any other way and still offer the convenience and speed that agency personnel needed. After a computer scientist joined the team, the first decision was to base the electronic model on the React JS framework software application, due to its platform flexibility. Using this system, and the questions already generated for each of the four outcomes, a prototype of the CRP Decision-Making Framework was created. In this prototype, all possible user responses to each question were tied through code to the appropriate recommendations. However, before the prototype could be tested, the team had to design and code an interface layout and format. A review mechanism also had to be installed before the tool would allow users to submit their responses to the framework questions. Soon after the first use of the prototype, the team recognized the impracticality of tying Output #2 (tools ) and Output 3 (people) to the same weighted process used for Output #1(whether to use a CRP, and, if so, how formally) and Output 4 (when to commence the CRP). The decision was made to drop the weights for Outputs 2 and 3, and simply link user responses directly to model recommendations. Once the process of tying responses to recommendations was complete, the prototype was re-coded to reflect these changes, and the user interface and some of the questions were refined.

209 The question data structure was then modified to accommodate multiple outputs. Soon thereafter, the team developed a more advanced prototype that would produce all four outputs (recommendation sets) under the newly developed and coded weighting and linking. Further testing of the prototype led to a fine-tuning and normalization of the weights for Outputs 1 and 4. The team was not satisfied with the reliability or robustness of the recommendations emanating from the model, because the weights tied to each question were based on solely the opinions of the eight team members. At that point, the team made two decisions: 1) the workshop, originally planned as an introduction of the latest model prototype, would also be used to gather input from the participants to strengthen and test the prototype; 2) the AHP would be used to gather data on which to base the weights for Outputs 1 and 4. In this setting, the AHP would force the participants to compare the importance of the answer to one question, to that of the answers to every other question. Once those decisions had been made, the team developed an electronic AHP model, and began preparations for the workshop. Preparing the AHP application required the integration of a random number generator to prevent any bias that might be generated from consecutive questions. Its use guaranteed that no two participants would have the same consecutive questions. A new heading page, and question-pairs had to be developed. The AHP model was based on the same (React JS) framework software used for the main CRP Decision-Making Framework model. Once this model had been developed and tested, each of the team members filled out the AHP questionnaires—one for Output 1, and one for Output 4. While this testing made them confident that the AHP process did indeed make the model more robust and accurate, they were also sure that the inputs from the many workshop participants would give it an additional boost. Several improvements to the AHP and CRP Decision-Making Framework models were necessary before they would be ready for workshop use. First, the “Load” and “Save” functions were developed and added to the AHP, to make it easier for the workshop delegates to answer questions and store their responses and results. The team then verified the accuracy of the AHP module. Similarly, the database structure and server had to be developed to a higher level to ensure the safe storage of responses and results. The team also upgraded the CRP Decision-

210 Making Framework model from a stand-alone desktop application to a server-hosted application. Finally, just before the workshop, the team conducted a dry-run of the CRP Decision-Making Framework, focused on troubleshooting connection issues and problems with database concurrency issues. During the Workshop, more than 40 constructability experts from around the country filled out the AHP questionnaires, enriching the accuracy and robustness of model with the benefit of their many years of experience. Each delegate was given an opportunity to operate the CRP Decision- Making Framework model, and give their opinions on its ease of use, accuracy, and helpfulness to their CRP decision-making processes. Using their comments, questions, and recommendations, the team then set to work putting the finishing touches on the model. In order for public transportation agencies (PTAs) to make the best use of the framework, the team decided to design it such that an agency can change certain parts of the code to fit their specific situations. Therefore, a guide was developed showing agencies how to make changes to parts of the model. While the weighting and the meaning of the questions and answer-options cannot be changed, the nomenclature and other aspects of the questions and answer-options can. For instance, a $30,000,000 project would be a large project for some PTAs, but not others. So, an agency can change the project cost ranges in the question that seeks to determine project size. The team implemented a function that allows the user to upload a custom file to the CRP framework. It also implemented a similar uploading function in the CRP editor, as well as a “Save” function to keep the custom file for future edits or use in the CRP Decision-Making Framework. The advantage of these functions is that many changes can be entered into the coding at once, instead of making the changes one by one. Another aspect to the model’s training materials was developed next. Pop-up messages were added to the CRP Decision-Making Framework model to provide guidance and clarification on certain questions and answer-options. In order to keep the pop-ups brief, links were added to some pop-ups, as needed. As with the pop-ups themselves, the user decides whether to use the links or not. Figure 13 shows a pop-up message clarifying a question about the LOD footprint of

211 a project. Finally, the team added a “Print” function and made a few aesthetic changes before submitting the fully developed CRP Decision-Making Framework to the panel, along with the rest of the project deliverables. Figure 13. A pop-up message clarifying a question about the LOD footprint of a project, superimposed over the model input questions. To use the CRP Decision-Making Framework, the PTA end-user should take the following steps in chronological order: 1. Enter the Project Description i.e., Project Title. 2. (Optional) If an existing local custom CRP file is available, Press “Load Custom CRP file”; choose the relevant custom CRP file; press “Open.” Your customized CRP framework will be loaded. 3. Press “Continue” to begin answering the CRP framework questions. 4. Fill in answers to all questions in each section before advancing to the next, by pressing “Next Section.” For more information and guidance about a particular question, press the “Information” icon located on the right. • A pop-up message will appear. • To return to a previous section, press “Previous Section”

212 5. Once all questions are completed, press “Review Answers.” A summary of all questions and their responses will be displayed. • To go back and change the answers, press “Edit Answers.” • To save a copy of the response, press “Print Answers.” • A system print window will appear. • Select the necessary settings to either save as a PDF file, or print out a physical copy. • To generate results from the response, press “Submit Answers.” 6. The ‘Results’ page will be displayed. Outputs 1 to 4 will be listed sequentially. • With regards to Output 3, people highlighted in blue are “in house,” whereas those highlighted in red are outside representatives. • For more information about a particular output or item, press the “Information” icon located on the right. • To save or print a copy of the results, press “Print Results.” A system print window will appear. • Select the necessary settings to either save as a PDF file or print out a physical copy. 7. To go through the process again for a new CRP, press “Start Over.’ It should be noted that the people listed in Output 3 are people recommended to participate in the CRP as needed. For instance, in the case of a historic building, cemetery, or bridge, the framework model may recommend that someone with knowledge of history participate in the CRP. That does not mean that this individual is recommended to participate in every meeting— just the meetings at which decisions will be made on how to handle the historical aspect of the project site. It is conceivable that this person may only need to attend one meeting. TASK 2-2: THE WORKSHOP The workshop was presented in 4 days over an 8-day period in 2.0- to 2.5-hour sessions. Invitations were issued first to the members of the NCHRP 10-99 panel, and then to individuals from the Phase I interviews, focus groups, and case studies. These invitees informed the team about other individuals who were knowledgeable and experienced in constructability. Since it would be held virtually, the team saw no reason not to invite anyone knowledgeable on the subject. Attendance for all the sessions ranged from 49 to 67 participants. (See Appendix C for the workshop agenda).

213 The purpose of the Workshop was four-fold: 1. Elicit information and recommendations to strengthen and increase the robustness of the CRP Decision-Making Framework. 2. Introduce the framework to individuals from DOTs and other transportation agencies from across the country (and internationally). 3. Identify a “champion” within each DOT represented, to encourage framework use in the home agency upon its release. 4. Educate the delegates on the most important progressive CRP practices identified through this research. The team went into each session with particular sets of questions. For the first session, held on Wednesday, March 3, the questions were as follows: • Why is Constructability important? • What is the difference between a PR and a CR? • What is the optimal time to start a CRP, and at what points should subsequent reviews be conducted? • What is the best way to get contractor input to the project design? The second session, held on Thursday, March 4 included the unveiling of the CRP Decision- Making Framework electronic model prototype. The session addressed the following questions: • Does the prototype of the framework suit the needs of DOTs? • What format and content would be most helpful and efficient for DOTs? o Input process? o Output? • How should Training Materials be presented? o Format? o Content? The third session was held on Wednesday, March 10, and included an exercise prompting all the delegates to utilize the AHP model, and give their input on the weights assigned to the framework model questions for Outputs 1 and 4. The questions for that session were as follows: • How can augmented reality and VR best be used to improve the CRP? • What are the proper weights for questions in the CRP Decision-Making Framework? The Thursday, March 11 session wrapped up the workshop. The delegates only answered one question that day:

214 • How can DOTs make their relationships with the FHWA stronger and more trusting, while at the same time, improving the environmental permitting and NEPA processes? For each session, the questions had been strategically aligned with the speakers, exercises, and anticipated outcomes. The objectives for each session were as follows: Session 1 • Educate participants on the purpose and status of the research. • Stress the importance of constructability and its timing. • Stress the difference between a PR and a CR. • Introduce the concept of the “Industry CRP.” Session 2 • Educate participants on the purpose and nuances of the proposed decision-making framework. • Demonstrate the prototype of the framework. • In the breakout session, have participants load actual past project information into the model, and gather feedback on the following: • The format of the output (is this type of information helpful?). • The perceived accuracy of the output. • Gain information on the format and content of training materials that would be most helpful and efficient to PTAs. Session 3 • Through the AHP, procure a significant quantity of data to give the framework model a higher degree of robustness and accuracy. • Educate the participants about the usefulness of augmented and virtual realities to the CRP. • Synthesize the feedback participants give on the industry use of augmented reality (AR) and VR in the CRP. Session 4 • Educate the participants about two creative ways that VDOT has taken their CRP to a new level • Opportunity to procure more data to give the framework a higher degree of robustness and accuracy. Breakout sessions were used to elicit information and data from the delegates. The breakout sessions had four to seven members, and each was facilitated by a research team member. Scripts were created to help these facilitators lead the discussion in a uniform manner. These breakout

215 sessions were the only opportunities for the delegates to get to know each other, and all of them participated in lively conversations on the discussion topics. Examples of the breakout session scripts and the entire workshop agenda are also presented in Appendix C. Speakers In addition to introducing and strengthening the framework, the workshop invited six speakers to share their perspectives on constructability: • Kim Lesay, Connecticut DOT, “Benefits of a CRP, and the Importance of Timing” • Jay Heitpas, Minnesota DOT and Sara Snow, Utah DOT, “Use of Contractors in Constructability, Pre-Bid Letting” • Angel Deem, Virginia DOT, “The Merged Process” • Scott Smizak, Virginia DOT, “Optimal Limits of Disturbance” Team Member, Dr. Raymond Issa and Dr. Ralph Tayeh also made a presentation called “The Use of Virtual and Augmented Reality in the Constructability Review Process.” Synopses of the six speakers’ presentations follow below. Kimberly Lesay – “Constructability, Permitting, and NEPA” Kim Lesay is Chief of the Bureau of Policy & Planning at ConnDOT. She began her presentation by having the delegates go through an abbreviated version of the CRP that ConnDOT executed twice over the last 20-plus years, in its effort to agree on a location for a bridge to replace the Stevenson Dam Bridge over the Housatonic River on Highway CT-34, near Lake Zoar, and the towns of Oxford, and Monroe, Connecticut. Most of her presentation dealt with this exercise, which took the delegates through the process that ConnDOT decision-makers used in the 1990s and then again in 2008. The main points of discussion, all related to the Stevenson Dam Bridge Replacement project, were as follows: • The public concerns • The 1990 design study • The 2008 VE study • NEPA risks

216 • Wetland permitting • Wildlife resources • Time-of-year restrictions • ConnDOT’s Planning and Environment Linkages process (especially as it pertains to constructability. Jay Heiptas – “MnDOT Industry Constructability Process” Jay Heiptas serves as Assistant Commissioner of Operations for the Minnesota DOT (MnDOT). He described MnDOT’s innovative process for involving contractors in the CRP. He gave the following reasons MnDOT conducts its industry constructability process as a meeting to discuss risk with potential contractors: 1) To reduce bidding risk and cost. 2) To decrease construction time. The contractors participating in the MnDOT Industry CRP may and, indeed are expected, to bid on the projects they review. VDOT is in the midst of adopting the industry constructability process, and the procedure for implementing it is described in the HRBT Case Study in Chapter 2. Sarah Snow – “Constructability Review Process and Platform Development in Preconstruction” Sara Snow is a consultant retained by the UDOT for her expertise in the CRP. She began her talk by listing the Top Five Benefits of CRs: • Increased safety for contractors and public • Significant reduction in COs • Millions in yearly cost savings to projects • Early design issue identification/reduction in rework • Project schedules enhanced/reduced in construction delay claims. She then discussed the five most-used tools in UDOT’s CR Tool Box: • Staff augmentation contract with team of former local contractors • UDOT Project Preview Program

217 • Constructability and estimating pool contracts • Alternate Delivery (CM/GC and PD-B) • UDOT construction crews and inspectors. The main benefit of the department’s staff augmentation contract with a team of former local contractors is the ability of team members to remain independent in their reviews. Moreover, it allows the CRP to start at the conceptual stage. She stressed the importance of the ex-contractors being local to the project location, and that early involvement is critical. Other benefits of this approach were listed as follows: • Instant access and availability. • Reviews performed at project milestones. • Innovations are identified, so the department can recognize the cost savings on bid day. • Risks are proactively identified and mitigated. Examples of items considered during a review by these former contractors include the following: • Materials Take-offs • Limitations of Operations and impacts to project durations and cost • ROW needed for construction and staging • Project phasing and staffing • Production-based cost impacts and constructability given specification requirements • Project construction durations • Utility conflict identification (room needed to do the work) • Material source locations and cycle times. The UDOT Project Review Program is a way that the departments are able to involve active contractors in their CRP. Unlike the Minnesota model, the UDOT process keeps the identities of the contractors anonymous, since it is all done online. Details of the process include the following: • A listserve notification is sent out to primes, subs, and suppliers.

218 • Anonymous comments and question are submitted online, with a 72-hour turnaround time for responses. • All project information is in draft form and for information only. • The process is used for complex projects, new pilot project methods, and high risk projects. • Project team involvement is critical during previews. Ms. Snow reported great success using both of these programs, each saving millions of dollars and copious amounts of time. Angel Deem – “Virginia NEPA & Section 404 Merged Process” Angel Deem serves as Director of the Environmental Division at the Virginia DOT. She began her talk by describing the motivation and context that led VDOT to adopt the Merged Process. On several occasions, VDOT has successfully completed the NEPA process, but not necessarily with buy-in from other federal agencies. In short, they found that the NEPA process was easier to negotiate than some of the other environmental-permitting processes. In these situations, following issuance of a NEPA decision and procurement, the design-builder was unable to move past the permit application on the preferred alternative. An enhanced effort to coordinate the NEPA and other permitting activities was needed on federally funded projects requiring a NEPA alternatives analysis and likely requiring an individual water quality permit. In the context of several project delays, VDOT began moving toward establishing the Merged Process, when it recognized the overlapping requirements of NEPA and Section 404 of the Clean Water Act. Virginia executed a MOU with FHWA in 2017 to establish the Merged Process. The stated purpose of the Merged Process is to “provide for a more efficient review of proposed highway projects by encouraging early communication among the agencies, provide structure to the review process, and identify general timeframes for components of the environmental review process, enabling VDOT to proceed with more transportation projects according to the

219 Commonwealth’s budget and identified schedule; and to ensure timeliness, the engagement of agencies, and the sustainability of decisions. The signatories of the MOU agreed on five “Concurrence” points in the Merged Process: 1. Environmental Analysis Methodologies 2. Purpose and Need 3. Range of Alternatives 4. Recommended Preferred Alternative 5. Conceptual Mitigation Among the Signatory Agencies are the following: • FHWA • U.S. Army Corps of Engineers • U.S. Environmental Protection Agency • U.S. Fish and Wildlife Service Among the Common Cooperating/Participating Agencies are the following: • U.S. Coast Guard • National Park Service • National Oceanic and Atmospheric Administration • Virginia Department of Environmental Quality • Local/Regional Planning Organizations Ms. Deem closed her presentation by having each delegate outline a Merged Process Agreement for their own state agency, or the agency with which their organization most often works. Scott Smizak – “NEPA Limits of Disturbance: Providing Flexibility for Future Design and Maintaining NEPA Schedules” Scott Smizak serves as Assistant Director of the VDOT Environmental Division and is involved in the execution of VDOT’s innovative LOD policy. The procedure followed by most PTAs is to make the footprint of a project’s LOD as small as possible, which allows them to coordinate with local environmental groups, and increase the probability of procuring a ROD. VDOT has found

220 that minimizing their LOD does not help to procure a ROD at all, and may lead to future problems. Planning-level LODs are used most frequently when generating EAs and EISs. Enlarging, even maximizing, the LOD for the project accommodates proposed improvements and future design changes. A study area for a project is developed with the idea of making it sufficiently large not only to capture any alternative, but also surrounding resources. Typical section(s) are developed for all options under consideration. For alternatives retained, a standard LOD is expanded beyond the typical section. Deviations to this practice are only considered to ensure de minimis use of Section 4(f) properties or when Section 6(f) properties can be avoided. Design criteria are presented to the agencies and documented early in the process, and the LOD is introduced to agencies and the public as a worst-case scenario. Routine discussions are then held with the pertinent agencies. All resources within the study area are mapped, documented, and discussed in the NEPA document and/or supporting technical reports. Impacts within the study area are reported for the LOD for each alternative. Technical requirements include NEPA impacts for wetlands, streams, protected properties, ROW, and other considerations. If the LOD is modified, the contractor accepts risk for obtaining necessary approvals. The optimized LOD expedites re-evaluations and design approvals. This is especially applicable and helpful when analyzing ATCs. Designs that are in compliance with NEPA LOD advance with limited documentation. This allows the design and construction teams to consider different technical approaches, and does not diminish VDOT Environmental’s review or approval of designs. Mr. Smizak then led the delegates through two case studies, before ending his presentation. Dr. Ralph Teyeh – “Constructability and Virtual Design & Construction” Dr. Teyeh, a post-doctoral student at the University of Florida, began his talk by describing BIM and the benefits of BIM: • Enhanced collaboration capabilities among all members of a project team

221 • Coordination of all building systems (clash detection) and the testing of design alternatives prior to construction • Ability to tie the model to schedules for visualization and quality assurance purposes (4D BIM) • Greater access to live data regarding building material quantities for more accurate cost estimates (5D BIM) • Creation of more accurate and thorough as-built documents • More effective project visualization at each project phase. He then described VR as “a three‐dimensional method to interface with computers, that allows the user to walk through the building model by wearing a head-mounted audio‐visual display, with a tactile interface device, position, and orientation sensors built in” (Kensek et al. 2000). His talk then turned to AR/Mixed Reality (MR), where he explained that AR overlays virtual objects on real objects. He went on to explain that a successful application of AR is a function of the coexistence of virtual and real objects in the augmented space, the registration of virtual and real objects with each other, and the running of a simulation in real time (Azuma et al. 2001). After speaking on AR, he moved on to Virtual Design and Construction (VDC), describing it as a completely new way of approaching projects supported by BIM, characterized by a new way to manage information and to organize people and their work methods. VDC combines the use of multiple technological tools, as needed. These tools include the following: • VR, AR, and MR • Reality Computing • Robotic Total Stations • Computer Science • Generative Design. Finally Dr. Teyeh discussed constructability issues, and the following constructability barriers: • Lack of Time – It is difficult to find a stage of project development—with plans complete— when there is time to get contractor input and make design changes based on that input.

222 • Lack of Manpower – Few organizations have sufficient personnel to staff the CRP. • Lack of Experience – Designers tend to lack construction experience, and contractors tend to lack design experience. • Contractor Reluctance – Some contractors fear they will lose their competitiveness by divulging proprietary construction means and methods to competing construction firms (Goodrum et al. 2003). The main constructability issues are: • Utilities – Trouble locating existing utilities before commencement of construction; construction delays due to utility relocation; unforeseen existing utilities that interfere with proposed construction; Existing utilities in locations other than shown on plans. • MOT/Traffic Control – Most current highway projects involve rebuilding and/or expanding existing roadway systems. Particularly problematic on bridge decks where clearance between traffic lanes and construction is often limited, due to the bridges’ physical dimensions. • Geotechnical problems – Unforeseen rock or soil conditions causing a need for COs. • ROW – ROW agreements not secured prior to construction thus causing delays; plans having insufficient detail during ROW negotiations; not enough ROW space for construction activities; schedule of securing ROW agreements not coinciding with the project’s construction schedule. He then reviewed how VDC enhances communication and collaboration through design, preconstruction, field construction, and handover & operations. Next, he described the leading software packages for selected applications, focusing on scheduling simulations. After delving into the advantages and disadvantages of different software packages, he discussed hidden measures of return for the use of BIM, with the emphasis on VDC. Finally, he considered the future of VDC in transportation construction, emphasizing its potential for supporting CRP. The Center for VDC at the M.E. Rinker School of Construction Management at the University of Florida conducted a survey of current construction professionals (contractors and owners). Citing the survey, Dr. Teyeh listed several applications of VDC, and showed percentages of each that the respondents rated as “now in use,” “will use in

223 1 year,” “will use in 3 years,” “will use in 5 years,” “will never use,” and “don’t know.” The applications were as follows: • Cloud computing • VR/AR/MR • Drones • Design for offsite construction • 3D printing of structural components • Analytics and big data technologies • Digital twins, sensors, machine-to-machine communication • Artificial intelligence, or machine learning. At least 30 percent of respondents worked for companies that already use the first four applications, but, at most, 11 percent of respondents work for companies that have ever used any of the last four. Main Research Takeaways When one considers the takeaways from the focus groups, case studies, and workshop, the theme most consistently repeated is that, if a CR is warranted at all, the review should start early. The recommended start time is before, or concurrent with, the beginning of the NEPA and environmental-permitting processes. Other important takeaways include the following: • DOTs have developed effective ways to involve active contractors and former contractors in the CRP. The Minnesota DOT has developed a process called the “Industry Constructability Review Process” that Virginia DOT is adopting. The HRBT Case Study in Chapter 2 presents the steps in the process. The Utah DOT has similar processes. The synopsis of Sara Snow’s workshop presentation above describes the benefits of the Staff Augmentation Contract with a team of former local contractors and the UDOT Project Preview Program (with active contractors). • The CRP challenges for which AR/VR can offer the most assistance are utilities, MOT/traffic control, geotechnical problems, and ROW. For details, see the synopsis above of Dr. Teyah’s workshop presentation above.

224 • The AHP proved to be effective in assigning weights to the issues involved in choosing the right people and the right tools for the CRP. • Virginia DOT has developed an approach combining the NEPA and other environmental approval processes that all but guarantees an ROD after its initial NEPA evaluation and the quick issuance of all needed environmental permits, while at the same time increasing the level of trust between the agency and the FHWA. (See Appendices B and C, and the HRBT/VDOT Case Study for details on the VDOT approach). • In order to understand a project well enough to maximize the benefits of the CRP Decision-Making Framework, project teams must know the core reasons for the project. • Senior DOT leadership buy-in and PM leadership increases chances of CRP success. • The use of internal and external technology and construction expertise will increase the impact of CRs at DOTs. • The value and benefits of the safety implications of constructability must be documented and consistently communicated to leaders and managers. • VDC using civil information modeling (CIM) and GIS tools can enhance project engineering, improve project management and reduce surprises during project execution. • Evaluating the long-term value of CRs is challenging, since virtually no empirical data have been gathered to identify or measure the cost and/or time savings realized from CRs. • The implementation of CRs varies greatly among state DOTs. Some DOTs (e.g., Minnesota and Utah) have specific guidance on how to conduct CRs while others may not rigorously conduct CRs across the PDP. • Industry must get the core team members of CRs involved early in the PDP. • DOTs should get “buy-in” from NEPA and other environmental-permitting agencies early, to minimize potential risk involved during the CRs and to enhance industry input and innovation. • Alternative contracting delivery methods (D-B, CM/GC) appear to facilitate the implementation of CRs by supplementing DOT projects with the essential staff, experience, knowledge, and team member coordination that may result in better CR performance and effective risk allocation.

225 • There is not only one way to implement constructability, since the DOTs and their project contexts vary. TASK 2-3: TRAINING MATERIALS The workshop had a major impact on the development of the training materials. One of the breakout sessions was dedicated to generating and compiling ideas for improving the framework and making the final product as simple as possible for agency personnel to use. In about 15 of the 70 recommendations, the participants suggested ways they and their people should be trained to use of the model. The training materials fall under two categories: 1. Definitions, explanations, and instructions for using the framework model. 2. Instructions on how to customize the model to fit an agency’s terminology and practices. After consulting with the workshop delegates, including the NCHRP 10-99 panel, the team decided to create a unique format for each of these two categories of training materials. The definitions, explanations, and instructions would take the shape of pop-ups or additional user guidance within the electronic model. These would address any terms within the input questions that may not be universally understood within the context of the model; the pop-ups would also define or discuss terms within the response options that may not be universally understood within the model context. When a user clicks on a question or possible answer, a pop-up message is likely to appear. Clicking on the pop-up message will bring up a message meant to explain the process or options available. In some cases, a single original term may be connected to multiple explanatory pop-ups or links. The user can pursue the thread as far as it goes, or until satisfied. For many terms and options, pop-ups are not needed. AIDS FROM INSIDE THE MODEL Model Inputs To help the user answer the questions with Likert scale ratings, the tool provides additional user guidance and commentary (instructions) for selected questions/directions as illustrated below. User Guidance Users can often understand a question on the surface, but may be unaware of the breadth and depth of the thinking that the model requires for best results. A click on the question’s

226 information icon can present them with factors to consider when formulating their answer. An example of this is the following: Question: Rate the environmental sensitivity of the project. Pop-up message: When rating the environmental sensitivity of a project, the project team should consider the following: • Potential for work within wetlands or waterways • Potential for noise pollution, particularly if the construction area is near residential areas or occupied buildings • Need to protect private property from runoff • Need to maintain existing storm drains • Potential for flooding • Potential of discovering hazardous materials, such as gasoline, or a landfill beneath the ground surface during construction • Proximity of the project to sinkholes and/or caverns. Definitions To alleviate any misconceptions about what is meant by certain terms in this context, definitions are provided. Following is an example of a term definition in the framework: Question: What is the project type? Answer: One of the options is “RRR.” Pop-up message: Resurfacing, Restoration, and Rehabilitation. Explanations Sometimes, the questions themselves need to be explained, and they too have pop-up messages. An example of this follows: Question: Rate the need to include people in the CRP, not for their potential input, but solely to increase their knowledge/experience in constructability. Pop-up message: Experience has shown that inclusion in CR sessions can be a valuable training and knowledge transfer exercise.

227 Outputs The tool processes the input information and provides recommendations in the following four output areas: 1. Necessity and formality of the CR 2. Tools to aid the CR 3. Composition of the CR team 4. Timing and frequency of reviews The tool provides additional user guidance and links that explain outputs; for example, levels of CR formality. End users should quickly become acclimated to the different outputs (recommendations) offered by the framework model, but new users will need some help— initially at least. The model offers assistance in the form of pop-up messages that the user must activate. In the case of the CR Formality output, although CRs are effective over a broad range of project types and sizes, the number and types of resources and effort needed for them can be substantial. Therefore, when devising a CRP for a particular project, the project team must strike the right balance between its potential benefits (e.g., refined project designs, enhanced construction efficiency, and reductions of disputes, cost overruns, and delays) and the time and effort it requires. The guidance presents three levels of CR formality: • Informal CRs: An informal CR mainly relies on project team experience and in- house inputs. The process itself may be largely ad hoc, with minimal use of analytical tools or relaxed adherence to prescribed policies and guidelines. • Semi-formal CRs: A semi-formal CR will generally incorporate more analytical tools and resources and will document a project’s constructability. • Formal CRs: A formal CR will strictly adhere to detailed policies/guidelines/procedures to comprehensively evaluate and document a project’s constructability. As conceptually depicted in Figure 14, the tools, resources, and frequency of reviews increases with increasing CR formality.

228 Figure 14. Levels of CR Formality Typical attributes associated with each CR level are further summarized in Table 3-3 to add further guidance for users.

229 Table 4 Attributes of CR Levels Again, hyperlinks to additional information are provided to help the user make the following decisions: 1) At which project stage should CR tools be used?; 2) Who on the project team or among project stakeholders should implement the CR?; and 3) Which references are most useful for guiding the user. Figure 14 presents the CR tools and show when they should be used during different project phases.

230 Figure 15. CR Tools and the Timing of Their Use in Different Project Phases. Below are three examples of the additional user information for CR tools used during the Constructability Planning stage. Most of this information is too lengthy to fit into the pop-up format, but can be accessed by clicking on a link inside a pop-up message.

231 Figures 16, 17, and 18 below present screen captures of the different kinds of pop-up messages in the framework model. Figure 16. A Definition Pop-Up Message on “Utility Mapping,” superimposed on the Model Input Questions.

232 Figure 17. An Instructional Pop-Up Message on an Input Question Superimposed over the Model Input Questions. Figure 18. A Pop-Up Message to Explain the Idea Behind a Question Superimposed over the Model Input Questions.

233 Constructability Planning 1. Checklists A consistent system of checklists is useful for minimizing the number of errors, inconsistencies, and omissions on construction projects. Checklists are a valuable means of minimizing oversights and errors, particularly for less experienced staff; and they help prevent the problems associated with last-minute fixes. Checklists serve as a means for the reviewers to focus on the areas and issues of concern (i.e., buildability of design, site investigation, ROW, staging, MOT, schedule). Several agencies have developed detailed checklists of 10 or more pages that have historically caused constructability problems, project delays, and cost overruns. When to Use: PIP through Construction Probable Users: Core Constructability Team; Designers, Construction Team References: https://mdotjboss.state.mi.us/webforms/GetDocument.htm?fileName=1960.pdf https://wsdot.wa.gov/sites/default/files/2016/05/16/ProjectDelivery- ConstructabilityReview.pdf 2. GPS (Global Positioning Systems). A satellite-based radio navigation system owned by the United States government. While it is principally a navigational system, GPS is also an important piece of mapping technology. Surveyors and others can use GPS to pinpoint the locations of objects to be shown on maps. GPS receivers installed on moving vehicles, for example, trucks carrying nuclear materials, allow them to be continuously tracked, and maps of their locations can be updated in real time. When to Use: PIP through Construction Probable Users: Core Constructability Team, Designers, Construction Team References: https://www.gps.gov/applications/roads/ 3. LiDAR (Light Detection and Ranging). Also known as 3-D laser scanning, LiDAR is a geospatial remote sensing technology used to make high-resolution maps. It is a method for determining ranges (variable distance) by targeting an object with a laser and measuring the time for the reflected light to return to the receiver. Lidar can also be used to make digital 3-D representations of areas based on differences in laser return times, and on varying laser wavelengths. Topographic lidar typically uses a near- infrared laser to map the land, while bathymetric lidar uses water-penetrating green light to measure seafloor and riverbed elevations. LiDAR has three primary applications: 1) Static LiDAR – system mounted at a single location to collect highly accurate data, but slower than other applications; exposes workers to traffic and other hazards. 2) Mobile LiDAR – system attached to a mobile object such as a vehicle or heavy equipment; uses LiDAR, inertial navigation systems, and GPS to measure roadway markings and cross sections. 3) Airborne LiDAR – system installed on an aircraft that can travel at speeds of up to 115 miles per hour at a maximum elevation of 1,600 feet. A key benefit of LiDAR technology is that its acquired data is useful for several applications. The data collected using LiDAR is analyzed in a digital point cloud format to map highway construction facilities and assets. Mining the collected data can produce suitable inputs for various construction-related design and modeling applications performed by DOTs, e.g., 3D and

234 4D digitized models and information for automated machine guidance. NCHRP Report 748 described various mobile LiDAR applications for delivering highway construction projects as follows: • As-built and maintenance documentation—Integration of LiDAR data into a centralized database that is continuously updated for future planning and construction, • Pavement smoothness and quality determination— LiDAR data collected at high resolutions can be used to evaluate pavement smoothness and quality, • Construction automation and quality control—Change detection and deviation analysis software uses digital models to identify deviations from LiDAR point clouds for construction quality control, • Performing quantity take-off—LiDAR data is used to calculate lengths, areas, or volumes of construction quantities, • Virtual and 3D Design— LiDAR data can be used for clash detection by checking for intersections of proposed objects with existing objects modeled in the point cloud, and • Inspections—LiDAR can provide overall geometric information and an overall condition assessment of various highway infrastructure assets (7, 11). When to Use: 30-percent CR through Construction Probable Users: Core Constructability Team, Construction Team References: Harper, C., Tran, D., Jaselskis, E., Implementation of Visualization and Modeling Technologies for Highway Construction: Current Practices and Future Trends, TRB, 2020. https://www.usgs.gov/special-topic/earthmri/science/topographic-lidar-surveys Olsen, M., G. Roe, C. Glennie, F. Persi, M. Reedy, D. Hurwitz, K. Williams, H. Tuss, A. Squellati, and M. Knodler, Guidelines for the Use of Mobile LiDAR in Transportation Applications, NCHRP Report 748, Transportation Research Board of the National Academies, Washington, DC, 2013. Yen, K., T. A. Lasky, and B. Ravani, “Cost-Benefit Analysis of Mobile Terrestrial Laser Scanning Applications for Highway Infrastructure,” ASCE Journal of Infrastructure Systems, Vol. 20, No. 4, December 2014. ADAPTATION GUIDE An agency may want to customize the framework model to make it better fit their terminology and practices. If so, users can make changes to the tool. Guidance on customizing the model takes the more conventional form of a chronological process. To customize the model, an end- user at a PTA should take the following steps:

235 1. (Optional) If an existing local custom CRP file is available, Press “Load Custom CRP file”; choose the relevant custom CRP file; press “Open.” Your customized CRP framework will be loaded for editing. 2. Press “Start” to begin editing. 3. Press “Edit Questions” to make changes related to the questions in the CRP. • Press “Edit Question” to make changes to a particular question. • Changes can be made to the question title, the answer-choices, the tools needed, and the people that each choice will lead to. • To edit the question title or answer-choices, select the corresponding editable textbox and make the required changes. • To edit the tools or people within a particular group of answer-choices, select the corresponding dropdown list and check/uncheck the boxes as required. • Press “Finish Editing” to move on to editing another question. 4. Press “Edit Tools” to make changes related to the recommended tools. • A list of tools will be displayed in editable textboxes. • To edit an existing tool, make the required changes in the corresponding textbox. • To add a new tool, press “Add Tool.” A newly generated item titled “New Tool” will be displayed at the bottom of the list. Make the required changes in the textbox. • To revert to the original state of the loaded/default CRP file, press “Revert Changes.” 5. Press “Edit Persons” to make changes related to the recommended people. • A list of people will be displayed in editable textboxes. • To edit an existing person, make the required changes in the corresponding textbox. • To add a new person, press “Add Person.” A newly generated item titled “New Person” will be displayed at the bottom of the list. Make the required changes in the textbox. • To revert to the original state of the loaded/default CRP file, press “Revert Changes.” 6. Press “Save Changes” to save the edits made to a local file. A file explorer window will appear. Select a folder or directory to store the custom CRP file. TASK 2-4: ELECTRONIC PRESENTATIONS The research team created two PowerPoint files to complement the training materials and, though it was assigned a different task number, it can be considered part of those materials. The presentations’ main goal is to help the agency personnel successfully execute the CRP Decision-

236 Making Framework on a proposed project. The team developed two versions of this presentation, one long and one short. The long version is the one described above. The short version is simply an abridged version of the same presentation, containing no additional information. The two are meant for different audiences. The longer version is for people who will be operating the system, while the shorter version is for agency executives, whose support is crucial to the success of the framework. The first six slides give an overview of the research project, and the history of constructability. The next 11 slides describe Phase I of the project, the literature review, the definition of the current state of the practice, and the development of the Conceptual Decision-Making Framework. The remainder of the presentation addresses the execution of Phase II tasks, and the subsequent analysis and use of the data gathered from the tasks. These tasks were 1) developing the CRP Decision-Making Framework, 2) conducting the workshop, and 3) developing the training materials. The Training Materials section includes an overview of the guide for PTAs to customize the CRP Decision-Making Framework model to better suit their particular needs, situation, and agency culture. TASK 2-5: TECHNICAL MEMORANDOM The technical memorandum discusses key research findings and recommendations for both future research and implementation of the CRP Decision-Making Framework. Additional details of the research in summary form, can be found in the Executive Summary and Conclusions of this final report. Beyond the data gathered and the process followed to develop the CRP Decision-Making Framework, the most relevant findings of this research were discovered in Phase I and during the workshop early in Phase II including the case studies and the workshop. Although the framework model was meant to produce findings in its application rather than through its development, the team discovered a few notable findings during that process; notably, the team was surprised at the extent to which participants (users) had various interpretations of questions, answer-options, and specific terms in the electronic framework.

237 • VDOT has a process that can serve as a model for how DOTs can use the advantages of IPD and not lose time, money, or quality due to conflicts arising from outside input into the design (i.e., from the contractor or design-builder). VDOT bases its system equally on “Optimized Limits of Disturbance” and something they call the “Merged Process.” A detailed discussion of how all DOTs can use this two-pronged system to improve their CRPs is a highlight of this report. Details of this system can be found in the HRBT/VDOT Case Study, and Appendices B and C. • DOTs should start their CRPs as early as possible. In state after state, researchers heard “one doesn’t need a set of plans to perform a constructability review.” The interviewees in the states most progressive in this area said they thought the CRP should start at the same time as the NEPA and environmental-permitting processes, or earlier. • DOTs should get “buy-in” from state and (especially) federal permitting agencies as early as possible. Some states meet with these agencies before beginning the NEPA and environmental-permitting processes, to familiarize these agencies with the scope and geographic limits of their projects. Sometimes they have “10-15-percent” plans to show the agencies, and sometimes they have no plans at all. After explaining the scope and showing them the project geometric footprint, they ask the agencies to advise them of any problematic facets or portions of the project that would prohibit permit application or issuance. This early “buy-in” from the agencies seems to make the permitting process go much more smoothly. • Submitting an oversized project footprint to the FHWA for the NEPA process has two advantages: 1) rather than exceeding the submitted LOD, which causes a NEPA re- evaluation, projects almost always require less land. This means a lesser environmental impact and smaller need for eminent domain land seizures than had been feared; and 2) submitting the larger footprint and using less than submitted improves the relationship between the DOT and the FHWA. The FHWA officials appreciated that VDOT officials provided an oversized project footprint rather than trying to claim the smallest piece of land it could in its NEPA proposal. By submitting an oversized project footprint, FHWA may be less concerned with small overruns so if the final design ends up requiring a small piece of land outside the footprint submitted in the NEPA proposal, FHWA may not require a NEPA re-evaluation.

238 o 38 survey respondents said their DOT has a formal CRP. o 25 respondents said their agency’s formal CRP changes when the delivery system changes. o 43 respondents said their agency’s effort to improve constructability involved/included industry input. o 26 respondents said their agency has established processes allowing contractor input into the design. o 12 respondents said they had witnessed project parties hampering contractor input into the design, out of fear of having to endure a NEPA re-evaluation or a reopening of the environmental-permitting process. • Minnesota DOT has generated guidelines for their CRP, into which they have inserted a 12-step process for gaining contractor input into the design. They have labelled it the “Industry Constructability Review Process.” VDOT is adopting the same plan. The process involves inviting contractors into the DOT office to view and analyze contract documents of upcoming projects early, before the projects are advertised. In exchange for reviewing the plans and specifications early, the contractor agrees to meet with DOT Construction and Design personnel for one hour to discuss constructability issues. • Of those interviewed in various states, several called for a formalized cross-training program. One stated that such a program would “help constructability more than just about anything we can do.” • District Design offices in the Florida and Connecticut DOTs have implemented a “Plans in Hand” Review somewhere between the 60- and 90-percent plans stage on all D-B-B jobs. This sometimes includes a field walk. They report having identified major problems at this meeting. Other DOTs have a similar procedure. • Over the course of developing the electronic decision-making model, the research team considered various forms of delivery such as a stand-alone desktop application or as a website application hosted on a web server. The team determined that hosting the application on a website would be the most accessible and convenient, since software downloads would not be required. The only requirement would be a modern web browser. The team observed that the installation of the stand-alone application added a layer of complication that negatively affected user experience.

239 • During the virtual workshop, the team faced issues with concurrency and internet connection while the participants were submitting their AHP forms. So, changes were made to the database system to handle multiple submissions at the same time, and an automatic retry function was added on the user side, to handle cases of initial submission failure. • The team also observed that when users had interpretations of the framework questions, answer-options, and terms that were different from the research team’s original meaning, model performance was negatively affected. To mitigate this problem, short pop-up modals were added to provide content clarification and user guidance. This supplemental information reduces confusion among participants without complicating the interface. Recommendations Recommendations those for implementation and future research. Recommendations for Implementation The limitations of this research indicate the need for additional research and development, and strategic, purposeful implementation procedures. (See the Recommendations for Future Research section below.) The NCHRP 10-99 panel incorporated specific tasks for creating training materials. While these elements are prepared for implementation, they require testing and refinement, similar to how the model itself needs ongoing improvement. The research team initially presented a prototype of the model to the workshop participants in March 2021. Since then, the research team has made numerous changes to enhance its functionality. Because of the changes, the status of the model is best described as a more powerful, robust, and comprehensive prototype compared to the one shown in March 2021. However, it is still in the prototype stage and requires evaluation in real-world conditions, akin to how the model needs to be tested and assessed in various scenarios to ensure its effectiveness. This indicated that there has been significant progress and improvement in the development of the model, but further testing and fine-tuning are necessary before it can be considered a fully operational and reliable tool. Just as with any evolving project, the process of refining and perfecting is ongoing.

240 The research also revealed the need to develop methods to measure the performance, not only for the model, but for an agency’s CRP. Such metrics are needed to allow continuous improvement. Topics suggested for consideration for implementation include the following: 1. Test and deploy training materials. All training materials developed as part of this research were developed after the workshop. Thus, none of them have ever been tested by practitioners, so the scope, funding, and time for the research did not include testing or implementation of the training materials. Much of their content is embedded in the model, so a thorough testing of the model would also test them. However, many of these materials are outside the model, and need testing separately. The research team believes a systematic and comprehensive approach to testing their effectiveness as selected DOTs implement the CRP Decision-Making Framework model on real projects would be helpful from the five or six states that participated in this research effort. The implementation of the training materials could include the participating states in this research effort to test and refine training materials for use by all PTAs. 2. Test the CRP Decision-Making Framework implementation on pilot projects. While use of the framework is available to all PTAs; only a few would know how to use it. Testing of the model on pilot projects and publishing the results as case studies would further the speed and effectiveness of model implementation. Since some of the training materials are embedded in the model, they could be tested at the same time as the model. The same researchers could test the guidelines for the purposes of customization right before or right after testing the model and training materials. This approach will allow refinement and prepare the model to be used by all PTAs. 3. Create, test, disseminate, and employ CRP performance measures. The research revealed that there was not a standardized approach to measuring the effectiveness of an agency’s CRP. If such a standardized, comprehensive CRP performance measures and targets were created, an e-model, rather than a “one-size-fits-all” approach would be more helpful. As with the CRP Decision-Making Framework, this e-model would ideally have mechanisms to allow agencies to customize the CRP metrics to their own needs. In the course of this research, the research team

241 gathered data for developing such a model for measuring the effectiveness of a CRP but more data would have to be gathered and amalgamated with the current data to develop the model. In addition to offering improvement of the CRP for all PTAs, research to develop a common set of CRP performance measures and targets would likely expedite and improve implementation of the CRP Decision-Making Framework. 4. Develop more electronic models similar to the CRP Decision-Making Framework, to meet agencies’ needs. During this research, the research team identified a potentially helpful procedure for developing electronic models (developing software) to help mitigate problems faced by today’s PTAs. As described in the preceding paragraph, a process similar to the one followed to develop the framework can be followed to create tools to help with emerging processes. Some areas to consider include the following: measuring the effectiveness of the CRP; the project closeout process (including a punchlist protocol); project bundling strategy; risk management; how best to deploy alternative contracting methods; quality management; optimization of LOD; and work packaging. Researchers could work with agencies to develop practical tools for any of these issues to meet the needs of agencies. These recommendations constitute four distinct opportunities to accelerate the diffusion of knowledge and the standardization of practice. Considerations for Future Research These considerations are based on the research, but due to timing, were not included in the CRP Decision-Making Framework. They also reflect the limitations of the research. 1. Future research on the reasons for a construction project would be helpful. Many experts in the focus groups, case studies, and workshop said that they found it difficult to use the framework unless they understood the reason the project was being built in the first place. It is not enough, they say, to know facts and statistics such as project size, project cost, and project duration. They need to know the core purpose for the project. How the purpose of the project affects the CPR should be integrated into this framework.

242 2. The current framework is not responsive to the delivery system being used or considered for a project. Therefore, if the framework were revised, it would be more sensitive to the delivery systems being used or proposed. Data pertaining to newer delivery systems, such as IPD and PD-B could also be included in the framework. 3. Contractors sometimes have issues when the highway or bridge they are building crosses or runs adjacent to a military installation. Research into how this situation affects the CRP, and integrating the knowledge procured into the framework would be helpful. 4. The current framework addresses availability of skilled and unskilled labor in the vicinity around the project site. Research into how to handle the issue of availability of equipment and materials (especially concrete) in the vicinity of the project site would also be helpful. 5. It would strengthen the framework to document the relationship between the CRP and Partnering, and to integrate that into information entered into the framework. 6. More research on the utilization of contractors in the CRP could help make the practice more widespread and more easily integrated into the framework. 7. The case studies afforded the team an exploratory understanding of the current state of CRP practice in the U.S. In the case study schedule for this project, no DOT from a state in the Southwest or Midwest was studied. 8. Add an artificial intelligence component to the CRP Decision-Making Framework, so the model can receive new data and continuously improve in performance. CONCLUSIONS The research addressed the primary research question, “Can a framework that is scalable to all delivery methods, that complies with all statutory requirements necessary to mitigate impacts to the natural and cultural environment, and that considers previously agreed upon environmental commitments, without limiting design innovation, be developed to assist PTAs in evaluating savings (cost, time, or both) from using CRs, and industry input during planning, design and permitting?” The research produced a stand-alone CR process (CRP) Decision-Making Framework to help decision-makers in PTAs frame their CPR to best effect for any specific project. The framework was not originally intended to be electronic, but in the process of executing the second task, “Benchmarking the Current State of the Practice,” the team realized

243 that, to make the framework usable to PTAs, it had to be electronic. A computer scientist was subsequently added to the team, and the final product is an easy-to-use model that receives user inputs from system prompts, and delivers recommendations on four essential concerns of CRP development: 1) Whether or not to perform a CRP on this project, and if so, how formal a process should be performed; 2) Which tools to use in the CRP; 3) Which people to include in the CRP; and 4) When to begin the CRP, and how often to re-engage the process. The state of the practice review identified the following important points and facts, illuminating the current CRP setting in PTAs, and affirming the necessity of the framework: • VDOT has developed a process that can serve as a model for how DOTs can exploit the advantages of IPD and not lose time, money, or quality due to conflicts arising from outside input into the design (i.e., from the contractor or design-builder). • VDOT has also developed a process by which they “optimize” their LOD in the NEPA process. This system eliminates the need to add to the proposed footprint, speeds up the NEPA process, and fosters trust between the FHWA and VDOT. • During the time of the research, the President signed an E.O. that requires all DOTs to follow most of VDOT’s model. • DOTs should start their CRPs as early as possible. • DOTs should get “buy-in” for expected permits from state and (especially) federal permitting agencies as early as possible. • 38 survey respondents said their DOT has a formal CRP. • 25 respondents said their agency’s formal CRP changes when the delivery system changes. • 43 respondents said their agency’s effort to improve constructability involved/included industry input. • 26 respondents said their agency has established processes allowing contractor input into the design. • 12 respondents said they had witnessed project parties hampering contractor input to the design, out of fear of having to endure a NEPA re-evaluation or a reopening of the environmental-permitting process.

244 • Minnesota DOT has generated CRP guidelines, into which they have inserted a 12-step process for gaining contractor input into the design. They have called this process the “Industry Constructability Review Process.” • Interviewees from several states called for a formalized cross-training program to improve the CRP. • District Design offices in the Florida and Connecticut DOTs have implemented a “Plans in Hand” Review somewhere between the 60- and 90-percent plans stage on all D-B-B jobs. This sometimes includes a field walk. Implementation Issues The results of this research provide the potential to advance and standardize the CRP throughout the U.S. The electronic CRP Decision-Making Framework gives PTAs an opportunity to apply a CRP only to projects for which one is necessary, and to commit only the minimum resources required to achieve constructability objectives. As with any initiative, maximum benefits can only be realized if top management fully supports the plan; and advancement of the CRP will require a willingness of highway agencies to formalize their approaches and measure their effectiveness. Standardization will require all states to be educated about the framework, and buy into the concept of improving their CRP. Having all state DOTs agree to use the framework will do more than anything else to standardize CRPs across the nation. How long did it take to implement the AASHTO Guide for Design-Build Procurement and to develop a standardized process that is shared among states? Is there yet a process that is shared among the states? Difficulty in the implementation of CRP approaches and devices are not unlike those inherent to the implementation of any procedural adjustments across large organizations, public or private. DOTs must contemplate numerous issues when endeavoring to adopt and use this CRP Decision-Making Framework. Some of these issues are as follows: • The bane of every person who ever tried to change anything in a large organization: “We’ve never done it this way before.” • The absence of state-level training and guidance manuals to promote consistency. • The perception that the model is a one-size-fits-all tool. Actually, the model comes with guidelines that will allow any agency to customize the model for its use.

245 • The absence of performance measures and targets to measure the effectiveness and efficiency of the model. • Some people assume that this will be just one more “time sink,” and that their units do not have enough people or resources to pay someone to operate the system. Actually, the model is simple to operate, and entering the required data—assuming the data are available to the operator—takes less than 3 minutes. The model’s recommendation, once the data are entered, appears instantaneously. The performance of this model has exceeded the expectations of nearly everyone connected with the research project. PTAs seeking to implement the model will likely need support in doing so, without which, few will likely proceed and may be faced with the implementation challenges outlined above. Additional research, education, and training can alert more PTAs to the existence and viability of the model, increase the pace of implementation, and promote standardization, and efficiency.