Maintenance releases, patch management and software updates keep users' experience optimal. Staying ahead of security threats is vital. So what caused and who is responsible for events leading up to the reflex action by the superintendent of Fairfax County Public Schools, the 10th largest school division in the nation located in one of the United States' wealthiest counties, to end the launch of the division's Distance Learning Plan two days into the project because of load capacity and security issues?

That's what the twelve elected members serving on the Fairfax County Public Schools Board turned their attention toward Thursday, April 16, during their virtual Board Meeting. The first item on the Agenda read: "Distance Learning Update: Blackboard."

The night before, Superintendent Scott Brabrand canceled teacher-led instruction through Blackboard scheduled for that day and Friday, April 17, due to continued login and security issues with Blackboard Learn FCPS 24-7 and Blackboard Collaborate Ultra.

"Our teachers are ready and willing to do whatever is necessary to provide a robust learning experience for our students. …We have identified our problems, and today we are proposing solutions," said Brabrand.

Later, during the School Board meeting, Marty Smith, Chief Operating Officer, said, "We were able to identify [disruptive] students by IP address, and then work with those principals to then refer those students for any additional discipline as necessary."

FCPS twelve School Board members, Brabrand and his Leadership Team composed of Frances Ivey, Deputy Superintendent; Marty Smith, Chief Operating Officer; Maribeth Luftglass, Assistant Superintendent, Department of Information Technology and Sloan Presidio, Assistant Superintendent and Instructional Services Department and Tim Tomlinson, Chief Product Officer for Blackboard, Inc., attended the meeting virtually.

As the 3-hour portion of the School Board Meeting wore on, individuals provided different levels of insight.

Whereas Tim Tomlinson of Blackboard initially commented with a prepared statement saying: "I'm here this morning on behalf of our (Blackboard's) leadership team to apologize for the challenges experienced this week in moving Fairfax County students to fully remote instruction," later he added: "We had no indication, based on our years of experience running the software for thousands of customers that we would have this issue at the levels of load that was expected this week."

One hour into the meeting, Sloan Presidio said, "As a Leadership Team, we failed to properly train the staff. We failed to properly communicate the expectations, and we failed to properly monitor the implementation." Presidio referenced security concerns and misbehaviors exhibited by then anonymous individuals who logged into virtual live class sessions via Guest Log-In, not through the preferred, guide-lined and more secure pre-populated email address access.

Nearly two hours into the meeting, School Board member Megan McLaughlin (Braddock) said, "I remain shocked that the 10th largest school system in the country did not know, working with its vendor, to do a load testing. There is no getting around it. …I want to say that the IT failures, the lack of flow testing, the software updates that didn't occur, the fact that we're doing information to our schools to prepare for this as guidance, versus direct as mandates, do speak to a larger problem we have in the system."

A letter on the FCPS official twitter account to FCPS families dated April 17, read: "Distance learning will resume Monday morning. Blackboard continues to make the necessary upgrades to the FCPS 24-7 system to bring students back to a safer, more stable online environment." FCPS signed the letter.

Q&A: What Went Wrong and Why?

Q- School Board member Ricardy Anderson (Mason): What were the issues?

A- FCPS Leadership Team member Luftglass: The first time that we saw some slowness and some challenges had to do with logging in. We had a lot of simultaneous logins on Monday (the day before Distance Learning went live to students) and Tuesday... There were a couple of different issues (with Blackboard 24-7), one with the logging system... and then once people got logged in, it overloaded the application servers... With Blackboard Collaborate Ultra sessions, there are a couple of different ways that you can implement those sessions. You pre-populate those sessions with the specific student email addresses that you want to have in your session. That's in the guidance documents... developed in terms of creating those sessions in a secure way so that you don't have Guest Access... Documents were created, but they were not properly shared and properly highlighted in terms of the importance, and that's our fault...Anybody could use those links (Guest Access) to get into the session...We failed.

Q- School Board member Karl Frisch (Providence): Was any request made to Blackboard to do testing to make sure, beyond simply telling them what our load would be?

A- Luftglass: No, we did not. We relied on our vendor to do that testing.

According to Tomlinson and confirmed by Luftglass, until recently, FCPS generally updated its software once a year, taking a one-week downtime after the academic school year ended and before summer school started.

Q- School Board member Sizemore Heizer (Member-at-Large): Why (wasn't) the software update… just done automatically?

A- Tomlinson: The Fairfax County Team makes decisions about when and how the software is updated... This system actually hasn't been updated in two years. We recommend that all of our clients stay on the most current versions of our software. The version of the software that Fairfax County is running is three years old. Fairfax County has made the choice not to update that software."

Q- School Board member & Vice Chair Tamara Derenak Kaufax (Lee): What are your best practices in place to make certain that your clients are getting the most up-to-date product from you?

A- Tomlinson: We release major updates twice a year. We release patch updates as frequently as needed. The system that's currently running (FCPS') was released three years ago. It's had two levels of patch updates applied to it… There have been seven updates made publicly available to the software that have not been applied.

Every time Blackboard releases an update, it's communicated directly to the client by email and its availability published on the company's Support Portal. Today…over one hundred of our clients are actually now running on a stats version of our product that's hosted by Amazon Web Services in the cloud… That's the most modern version of our software that we've been recommending to all of our clients for a couple of years. (It) is updated automatically every month; no action required by the client… That's the best practice we deliver for software. It's a change that Fairfax County has not chosen to take advantage of over time.

Latest on Distance Learning: School System Looking for ‘Plan B’

Monday evening’s message to Fairfax County Public Schools, parents:

“Despite attempts to address identified technology issues over the weekend, this morning it became evident that Blackboard Learn 24x7 was not able to handle the system load of our virtual class meetings. This evening, Dr. Brabrand emailed the community with his comprehensive plan on addressing the problems that have arisen with the current distance learning format.

To ensure students have access to continuity of learning, students will resume asynchronous assignments tomorrow (Tuesday). Students should log into Google Classroom in order to access choice boards for this week. Staff will be available to answer questions via email for students needing further guidance.

We will provide you with an update soon regarding our next steps as we shift our distance learning plan. Please know that our teachers and staff have been working and adapting with every change to best teach our students in this new environment. As new plans are identified, the staff will continue to create assignments to engage students in meaningful learning activities.

We know that this is frustrating, and we appreciate your patience as we adjust to the new plan.”