In this webinar, James Governor, co-founder of Redmonk, will share his perspectives on the advantages of AI in software development. You’ll gain an understanding of the primary benefits of using AI coding tools, which include enhanced code quality, quicker onboarding with new codebases, improved security, and more. Eirini Kalliamvakou, Staff Researcher at GitHub, and Allison Weins, Staff Product Manager at GitHub, will also discuss how to develop organizational policies around generative AI. Learn effective frameworks and rollout strategies to support the adoption of enterprise-grade AI coding tools across your engineering teams.

Join this thought leadership webinar to explore the transformative role of generative AI in modern software development.

__Learn more about:__

- What the rise of generative AI actually means for developers and businesses.
- How AI is reshaping the traditional software development lifecycle, enhancing productivity, and accelerating release cycles.
- What enterprise development teams actually think about generative AI–the pros and the cons.
- Success stories with real-world examples of how AI-powered tools have helped companies overcome challenges and achieve new milestones.
- The future trajectory of generative AI in software development and how enterprises can prepare for upcoming changes.

AI has reigned as the topic of the day over the past two years. Its prevalence spans everything from billboards to executive briefings. And with tools like GitHub Copilot, it’s clear that generative AI is changing software development—but how? What do enterprise development teams think about generative AI? 

AI Background - Artificial Intelligence pillar - Dark 4

Resource Page for /artificial-intelligence/adopting-generative-ai-in-software-development

Adopting Generative AI in Software Development

/artificial-intelligence/ai-coding-tools

Explore the current and future impact of AI on developers—and find out why humans will always be essential to delivering innovative software. 

Read the Gartner® report, AI Will Not Replace Software Engineers (and May, in Fact, Require More), to:

- See how AI will transform the role of engineering teams today, tomorrow, and well into the future.
- Get recommendations—backed by research—about how to prepare for the emergence of AI engineering.
- Learn how the use of AI-powered tools will create distinct career paths for junior and senior developers.

Gartner, AI Will Not Replace Software Engineers (and May, in Fact, Require More), Philip Walsh, Gunjan Gupta, Helen Poitevin, Keith Mann, Dave Micko, Manjunath Bhat, 30 August 2024. 

---

*GARTNER is a registered trademark and service mark of Gartner, Inc. and/or its affiliates in the U.S. and internationally and is used herein with permission. All rights reserved.*

*Gartner does not endorse any vendor, product or service depicted in its research publications, and does not advise technology users to select only those vendors with the highest ratings or other designation. Gartner research publications consist of the opinions of Gartner's research organization and should not be construed as statements of fact. Gartner disclaims all warranties, expressed or implied, with respect to this research, including any warranties of merchantability or fitness for a particular purpose. *

Explore the current and future impact of AI on developers and see why humans will always be essential to delivering innovative software in this report. 

Resource Page for /artificial-intelligence/gartner-ai-will-not-replace-software-engineers

AI Will Not Replace Software Engineers (and May, in Fact, Require More)

Landing Page for /artificial-intelligence/gartner-magic-quadrant-for-ai-coding-assistants-2024

Read __How enterprise engineering teams can successfully adopt AI__ to explore how GitHub is at the forefront of AI innovation—and how it can help your organization lay the groundwork for successful AI adoption.

Get the ebook to:

- Learn how generative AI has gone from being niche to mainstream and get an overview of common AI-powered developer tools, including GitHub Copilot.
- Understand how AI is reshaping enterprise software development and assess the opportunities that your organization has to benefit from AI.
- Get a roadmap with step-by-step instructions on how to operationalize AI solutions for your engineering teams.


Learn how to bring AI to your engineering teams and maximize the value that you get from it.

Resource Page for /artificial-intelligence/how-enterprise-engineering-teams-can-successfully-adopt-ai

How enterprise engineering teams can successfully adopt AI

## The GitHub Copilot Impact Glossary
This glossary lists a number of measures that an organization can use to help evaluate the impact of Copilot for their organization. Learn:
- Definitions and caveats related to measures
- When (which stage) to use a measure
- How to access a measure (source data location)

For more information on GitHub’s approach to measuring Copilot Impact, check out our [GitHub Copilot Learning Pathway](https://resources.github.com/learn/pathways/copilot/essentials/measuring-the-impact-of-github-copilot/).

Remember 😀: The relevance and usefulness of system-level speed and quality measures vary across organizations. Any Copilot impact beyond coding is indirect, and the downstream possibilities are endless. Thus, we must be careful when searching for causality in speed and quality measures. 

The metrics provided below are representative of processes that often improve with Copilot, but the degree of impact will vary. Based on the strengths and weaknesses of your organization, the impact may be more or less. It's less important to achieve a benchmark than to observe neutral-to-positive improvement. Engineering leaders should only be concerned if a metric regresses. In that case, pause to determine the root cause before proceeding.

### Tips: 
- GitHub recommends looking for improvements in each measure over time rather than setting fixed targets.
- We suggest aligning measurement frequency with your development cadence (e.g. sprint cycles).
- You may choose to collate your data points into existing BI or reporting tools for ease of analysis and review over time for trends. Alternatively, use a spreadsheet to track measures over time.
- As you add developers to Copilot you may see a decrease in some measures as those developers build capability and confidence in using Copilot as part of their daily flow. Consider analyzing measurements according to onboarding groups (set of developers provided with a Copilot seat).
- You may choose to monitor some DevOps metrics over the evaluation and adoption stages for early indicators of likely system- impact areas. But particularly in the evaluation and adoptions stages, we recommend focusing on happiness and engagement measures. Once developers are using Copilot effectively, system level measures are more likely to improve. 

### Category legend:
Developer Happiness - 😃 
Developer Engagement - 👩‍💻 
Speed - 🏎️ 
Quality - ✨

<table>
 <tr>
 <th>Measure</th>
 <th>Definition</th>
 <th>Stage</th>
 <th>Source data location</th>
 </tr>
 <tr>
 <td>Satisfaction using Copilot - 😃</td>
 <td>Number of Copilot users who value Copilot in their workflow.</td>
 <td>All</td>
 <td>Copilot Impact Survey: • How would you feel if you could no longer use Copilot?</td>
 </tr>
 <tr>
 <td>Benefits of using Copilot - 👩‍💻</td>
 <td>Range of areas in which users leverage Copilot and the range/extent of benefits experienced.</td>
 <td>All</td>
 <td>Copilot Impact Survey: • When using Copilot, I… • When using Copilot, my team… • In using Copilot in the last week… • I use GitHub Copilot to…</td>
 </tr>
 <tr>
 <td>Challenges using Copilot - 👩‍💻</td>
 <td>Number of users impacted by potential adoption barriers.</td>
 <td>All</td>
 <td>Copilot Impact Survey: • What challenges have you encountered during your Copilot trial?</td>
 </tr>
 <tr>
 <td>Enablement provided - 👩‍💻</td>
 <td>Number of users with a seat who have received enablement resources/training sessions.</td>
 <td>All</td>
 <td>Copilot Impact Survey: • Which of the following GitHub Copilot enablement materials have you received?</td>
 </tr>
 <tr>
 <td>Average Daily Active Users per Month - Completions - 👩‍💻</td>
 <td>Users who have received 1+ suggestions on a given day.</td>
 <td>All</td>
 <td>Copilot Metrics API: • Take average of Active Users - Completions from API, over 28 days</td>
 </tr>
 <tr>
 <td>Average Daily Active Users - Chat - 👩‍💻</td>
 <td>Users who have 1+ chats on a given day.</td>
 <td>All</td>
 <td>Copilot Metrics API: • Take average of Active Users - Chats from API, over 28 days</td>
 </tr>
 <tr>
 <td>Average Daily Active Users - Total - 👩‍💻</td>
 <td>Unique users who have received and accepted 1+ suggestions OR participated in 1+ chats on a given day.</td>
 <td>All</td>
 <td>Copilot Metrics API: • Take average of Active Users - Total from API, over 28 days</td>
 </tr>
 <tr>
 <td>Suggestions Delivered - Completions - 👩‍💻</td>
 <td>Number of suggestions per day displayed in the code editor for Copilot completions in IDE.</td>
 <td>All</td>
 <td>Copilot Metrics API: • Number of Suggestions</td>
 </tr>
 <tr>
 <td>Number of Acceptances - Completions - 👩‍💻</td>
 <td>Number of suggestions accepted (total of both partial and full) per day.</td>
 <td>All</td>
 <td>Copilot Metrics API: • Number of Chat/code acceptances</td>
 </tr>
 <tr>
 <td>Lines of Code Accepted - Completions - 👩‍💻</td>
 <td>Lines of code accepted (total of both partial and full) accepted per day</td>
 <td>All</td>
 <td>Copilot Metrics API: • Lines of Code Accepted - Completions</td>
 </tr>
 <tr>
 <td>Total Acceptance rate - Completions - 👩‍💻</td>
 <td>Suggestions Accepted (Combined: Full and Partial) / Total Number of Suggestions</td>
 <td>All</td>
 <td>Manual calculation (based on Copilot Metrics API): • Number of Suggestions & • Number of Acceptances</td>
 </tr>
 <tr>
 <td>New seats added in billing cycle - 👩‍💻</td>
 <td>Additional seats purchased for organizational members during this billing cycle</td>
 <td>Adopt, Optimize, Sustain</td>
 <td>Billing and plans (Copilot) (organizational settings) • New seats added during this billing cycle</td>
 </tr>
 <tr>
 <td>Dormant users - 👩‍💻</td>
 <td>Number of users who last used Copilot more than ‘n’ days ago</td>
 <td>Adopt, Optimize, Sustain</td>
 <td>Copilot Access Management (organizational settings): • Sort by last used</td>
 </tr>
 <tr>
 <td>Total completed pull requests - 🏎️</td>
 <td>Total number of pull requests completed in a given week</td>
 <td>Adopt, Optimize, Sustain</td>
 <td>GitHub PRs list (in a repo) can be filtered on closed PRs for a time period</td>
 </tr>
 <tr>
 <td>Pull requests per developer - 🏎️</td>
 <td>Number of pull requests / code contributors</td>
 <td>Adopt, Optimize, Sustain</td>
 <td>Manual calculation: • GitHub PRs list (in a repo) can be filtered on closed PRs for a time period & • Your organization's data on developers contributing to repo</td>
 </tr>
 <tr>
 <td>Time to merge - 🏎️</td>
 <td>Average time from PR opened to merge</td>
 <td>Adopt, Optimize, Sustain</td>
 <td>GitHub: calculation requires consideration of your organization's GitHub set-up</td>
 </tr>
 <tr>
 <td>PR lead time - 🏎️</td>
 <td>Average time from PR opened to deploy</td>
 <td>Adopt, Optimize, Sustain</td>
 <td>GitHub Actions: calculation requires consideration of your organization's GitHub set-up</td>
 </tr>
 <tr>
 <td>Code Review Time - 🏎️</td>
 <td>Average code review time per pull request (time between PR opened to PR closed)</td>
 <td>Adopt, Optimize, Sustain</td>
 <td>GitHub: calculation requires consideration of your organization's GitHub set-up</td>
 </tr>
 <tr>
 <td>Pull request merge rate - ✨</td>
 <td>Total number of successful builds completed in a given week.</td>
 <td>Adopt, Optimize, Sustain</td>
 <td>GitHub: GitHub PRs list (in a repo): The number of PRs merged / The number of PRs submitted</td>
 </tr>
 <tr>
 <td>Total successful builds - ✨</td>
 <td>The percentage of pull requests successfully merged</td>
 <td>Adopt, Optimize, Sustain</td>
 <td>GitHub Actions: calculation requires consideration of your organization's GitHub set-up</td>
 </tr>
 <tr>
 <td>Change failure rate - ✨</td>
 <td>Percentage of code deployments that caused a failure in production</td>
 <td>Adopt, Optimize, Sustain</td>
 <td>Calculation requires consideration of your organization's GitHub set-up and integration with third-party apps</td>
 </tr>
 <tr>
 <td>Open security vulnerabilities - ✨</td>
 <td>Number of unresolved vulnerabilities as identified by GHAS</td>
 <td>Adopt, Optimize, Sustain</td>
 <td><a href="https://docs.github.com/en/enterprise-cloud@latest/code-security/security-overview/viewing-security-insights-for-your-organization#autofix-suggestions">Organizational-level security insights</a></td>
 </tr>
 <tr>
 <td>CI success rate - ✨</td>
 <td>Number of successful CI runs inclusive of test automation / Number of attempted CI runs</td>
 <td>Adopt, Optimize, Sustain</td>
 <td>Calculation requires consideration of your organization's GitHub set-up and integration with third-party apps</td>
 </tr>
</table>

We recommend to first focus on your developers’ adoption and engagement with Copilot. Then, harness developers’ improved productivity to pursue the engineering system improvements that matter most to your organization.

/artificial-intelligence/

Adopting Generative AI in Software Development

AI Will Not Replace Software Engineers (and May, in Fact, Require More)

How enterprise engineering teams can successfully adopt AI

Glossary: Measuring the impact of GitHub Copilot