Every program at DARPA is intensely technically scrutinized by the tech council
The Tech Council is composed of people with technical expertise in the proposed program’s area and adjacent areas. The Tech Council Pitch Meeting is meant to be very high level but council members can ask deep technical questions on anything. The tech council doesn’t have any power besides advising the director on the program’s technical soundness. A purely advisory tech council seems like a good idea because it both avoids decision by committee and keeps all responsibility squarely on the director and PM.
I see this as something like an external audit. I am a big fan of audits. The essay I am working on will suggest audit mechanisms as a way out of our “post-truth” morass.
the closest thing to a framework that PMs use to guide program design is “be able to explain precisely why this idea will work to a group of really smart experts both in the area of the program and adjacent to it.”
Imagine if every research project or graduate course were put through this sort of process.
Pointer from Tyler Cowen, who correctly lauds the essay.
“Imagine if every research project or graduate course were put through this sort of process.”
This is why it would likely never happen.
“ I am a big fan of audits.”
Having worked as an auditor, I have to laugh at this one. It isn’t a coincidence that another one of the big international auditing firms (EY, Wirecard), is at risk of collapsing. External auditing is mostly about figuring out the very bare minimum of work needed to be done to have a good chance to prevail in potential litigation.
That said, so far the PCAOB seems to fulfilling its intended role in the US.
But the fact that external audits aren’t reliable without a government agency who continually examines and continually brings disciplinary action against external auditors I think speaks volumes about what how little be expected of external audits absent such oversight and prosecution.
As a former auditor myself, I think you’re a bit overly cynical, here. How many of the failures you’re describing are inherent limitations of independent audits? How many are due to the oligopoly that the Big Four represent enabling them to act like rent seekers? I would say there’s a little from column A, a little from column B, at the least.
I would also sadly add that things seem even worse to me in the government world, where I am now. Medicare and Medicaid hire a myriad of third party vendors to do “audits” of this, that and the other thing, but which aren’t really audits at all, as the term is understood in the private sector. Invoking the word seems to be done, though, to add a veneer of confidence and independent oversight to the process of shoveling public money out the door. It’s a sorry state of affairs, really. So while maybe your cynicism about the private sector is justified, it’s worse in the public sector, at least in my arena.
All large organizations I have encountered are big fans of Project Management frameworks. Proper certification is very often a requirement for the Project Manager. None of these organizations are even remotely serious about having strong, semi-independent auditing in their projects.
For a brief period the Dutch government had a task force dedicated to auditing large IT projects that was so successful in finding mismanaged projects that they were briefly celebrated and quietly shutdown.
Any chance you can post a source where I could find out more on that? Would love to read about it.
This translated article should not paywalled and serves as a general guideline to the controversy
https://translate.google.com/translate?hl=en&sl=nl&u=https://www.nrc.nl/nieuws/2019/09/06/kamer-wil-ict-waakhond-behouden-en-beschermen-a3972427&prev=search&pto=aue
Audits generally imply transparency, but this guy is attributing DARPA’s success to it’s opacity. Given DARPA’s politicization ( see: https://www.foxnews.com/politics/dems-deploying-darpa-funded-information-warfare-tool-to-promote-biden ) It is probably not a good selling point for the “keep the little people in the dark and don’t let them have a say in anything.” DARPA also counts academia as a partner. Given academia’s fondness for General Secretary Xi and the overall success on USA campuses, one wonders how much DARPA funding is benefitting the CCP. I say the extant audit process is insufficient and greater transparency and accountability would be an improvement.