Reflecting on GSoC 2024

An exciting summer has come to a close for stdlib with our first participation in Google Summer of Code (GSoC). GSoC is an annual program run by Google and a highlight within the open source community. It brings together passionate contributors and mentors to collaborate on open source projects. Selected contributors receive a stipend for their hard work, while organizations benefit from new features, improved project visibility, and the potential to cultivate long-term contributors.

stdlib (/ˈstændərd lɪb/ "standard lib") is a fundamental numerical library for JavaScript. Our mission is to create a scientific computing ecosystem for JavaScript and TypeScript, similar to what NumPy and SciPy are for Python. This year, we were granted four slots in GSoC, marking a significant milestone for us as a first-time participating organization.

The purpose of this post is to share our GSoC experiences to help future organizations and contributors prepare more effectively. We aim to provide insights into what worked well, what challenges we faced, and advice for making the most out of this incredible program.

Highlights of the Program

While we certainly encountered bumps along the way (more on that in a second), overall, our participation in GSoC was packed with standout moments. Our accepted contributors successfully completed their four GSoC projects.

To illustrate the impact of our participation, here are some key statistics and accomplishments from our community since the GSoC organization announcement in February:

• Over 1,000 PRs opened
• More than 100 unique PR contributors
• Over 2,000 new commits to the codebase

We had a range of successful contributions that significantly advanced stdlib. Specifically, our four GSoC contributors worked on the following projects:

• Aman Bhansali worked on BLAS bindings, overcoming the challenge of integrating complex numerical libraries into JavaScript.
• Gunj Joshi developed C implementations for special mathematical functions, significantly improving the performance of our library.
• Jaysukh Makvana added support for Boolean arrays, enhancing the library's functionality and usability and paving the way for NumPy-like array indexing in JavaScript.
• Snehil Shah worked on enhancing the stdlib REPL for scientific computing in Node.js, making it easier for users to interact with our library and perform data analysis in their terminals.

Each project addressed critical areas in our mission to create a comprehensive numerical library for JavaScript and the web platform.

Finally, we already see a glimpse of the project attracting long-term contributors from both GSoC participants and the broader community.

An Unexpected Challenge

Despite the many positives, our journey wasn't without its share of challenges. Early on, we faced an unexpected incident that seemed straight out of a movie plot. A prospective contributor tried to sabotage a fellow applicant by impersonating them on Gitter, the open source instant messaging and chat room service where we engage with the community. After signing up via a fake Twitter/X account, he started sending unhinged messages to several of the project's core contributors. While it quickly became clear that we were communicating with an impersonator, it was an unsettling experience nonetheless. The impersonator even ended up copying the real applicant's proposal and later attempted to claim the work as their own on GitHub after the conclusion of GSoC.

In light of this experience, we advise any organizations participating in GSoC to keep in mind that competition for slots can be fierce, and that some individuals may be tempted to use subterfuge or actively jeopardize others' applications. One must be vigilant and expect the unexpected. We also recommend having a Code of Conduct (CoC) in place to address such unethical behavior and raising awareness among GSoC contributors of its existence, such as having a CoC acknowledgment checkbox on pull requests and when submitting proposals.

Lessons Learned and Advice for Future Participants

Engage Early with the Community

First and foremost, it is crucial to encourage potential contributors to start interacting with the community and codebase well before the application period. This helps build familiarity and commitment. Although we were aware of this, we could have done more to encourage early engagement and provide clearer guidance on how to get started. Going through all onboarding steps afresh may help uncover outdated information in documentation or other inconsistencies.

Handling Community Queries

After our participation was announced, we were quickly bombarded with what seemed like a non-stop barrage of messages per day on Gitter and other communication channels, and with dozens of PRs opened each day. As the core stdlib team is not working on the project full-time, it was very challenging to keep up. We learned that it's essential to set clear expectations and boundaries early on to manage the influx of new contributors.

Managing the Onboarding Process

Answering the same questions repeatedly can be time-consuming, so having frequently asked questions (FAQs) and a well-documented onboarding process will prove to be invaluable. We also started a weekly office hour for people to drop by. This had a decent turnout and proved valuable, as only individuals who were genuinely interested in the project attended and helped weed out those who were just making "drive-by" contributions. In addition to the weekly office hours, we also held two sessions during the application period to serve as informational sessions specifically focusing on GSoC so we could answer all questions that prospective contributors had.

After the conclusion of GSoC, we have continued to hold weekly office hours, which have been a great way to keep the community engaged!

Good First Issues

What worked less well were the "good first issues" issues we had opened and labeled as such on GitHub. We found that issues we thought were good first ones, such as updating documentation and examples, resulted in a very high number of low-quality submissions, often suffering from hallucinated contents due to AI generation or other issues, which caused more work for reviewers. On the other hand, other tasks, such as refining TypeScript definitions, were often too complex and challenging for newcomers.

We learned that the best first issues are those that are well-scoped, have clear instructions, and are easy to test and verify. Having a bunch of trivial issues provides weak signal; you want to see contributors progressively tackle more complicated tasks as they become more acquainted with the project. To aid in this progression, one would be well served to have enough issues of varying difficulty that prospective contributors can tackle. If possible, it may be ideal to have issues build on top of each other and take the contributor on a journey toward mastery. Similarly, it may be good to create open issues that are related to each of the potential GSoC project topics, so that contributors can get familiar with the parts of the codebase they would be working on during the GSoC program. And lastly, consider creating issue templates specifically for GSoC participants, which include detailed instructions, links to relevant documentation, and expected outcomes. This reduces ambiguity and helps set clear expectations for newcomers.

Going forward, we plan to focus on creating well-defined, incremental issues that serve as stepping stones for new contributors to build familiarity and gradually take on more complex tasks.

The Role of AI

I think it's fair to conclude that Generative AI has emerged as both a blessing and a curse in the world of open source contributions. Personally, I am an avid user of LLMs and happy about the innovation they have sparked in the developer tooling space. They can assist non-native English speakers in better communicating their ideas, provide a conversation partner equipped with vast knowledge of even quite remote topics, and can increase developer productivity through code completions and code generation. However, AI has also led to a flood of low-quality PRs generated by AI tools, often filled with hallucinated code or content that doesn't align with the project's actual requirements. While writing code can feel more rewarding than the often tedious task of reviewing it—especially when the code isn't your own—reviewer fatigue becomes a real issue when faced with a barrage of poorly constructed or misaligned PRs.

Contributors must recognize that AI is an assistant, not a replacement for personal responsibility and craftsmanship. We have by now spent a significant amount of effort in automation to filter out low-effort submissions before they even reach the review stage. Beside workflows that close PRs which don't adhere to basic contribution conventions, we have added jobs that post helpful comments on how to set up a development environment or which remind contributors that they have to accept the project's contribution guidelines before their PR can be reviewed. This significantly reduces the burden on reviewers and ensures contributors are aware of expectations from the beginning.

Contributor Triage

Another important takeaway is to watch out for contributors claiming multiple issues without completing them. We found that it's best to avoid assigning issues to anyone via the respective GitHub feature and instead focus on encouraging quality contributions over sheer quantity. Additionally, be prepared to manage contributors who may place unrealistic demands on review times, such as insisting on immediate feedback.

One has to be ruthless in prioritizing contributions. This approach ensures that contributors who show genuine interest and effort receive the attention they deserve, leading to higher quality interactions and outcomes for both the project and the contributor. Reviewer time is a limited resource, and it's simply not feasible to provide equal attention to every contributor.

At the end of the day, contributors must invest the time necessary to familiarize themselves with a project's conventions, guidelines, and best practices. If they don't meet this minimum threshold and do not show genuine effort, it's not worth allocating the finite resources of the core team. This may sound harsh, but it's necessary to ensure there is enough time to focus on the high-quality contributions. Otherwise, one ends up in a position where everybody is unhappy with your responsiveness. This may be less of an issue for organizations in niches requiring specialized skills and which may not have as wide an audience as a JavaScript library.

Provide Clear Documentation

Ensure that your project documentation is comprehensive and up-to-date. This includes installation guides, contribution guidelines, and a clear roadmap. Poor documentation can be a significant barrier to entry. During the community bonding period, we found that our documentation was outdated in some areas and that there were issues arising from our setup instructions not working on all operating systems. Providing a devcontainer setup for Visual Studio Code helped to mitigate these issues and streamline the onboarding process.

Mentor Selection and Training

Choose experienced and committed mentors who can provide guidance and support throughout the program. Consider providing mentor training sessions and setting clear expectations around time commitments and responsibilities to better prepare mentors for their roles. Expect mentoring to be more demanding than envisioned.

We found that having weekly stand-ups allowed contributors to get to know each other and share their progress. We had also, early on, decided to have weekly 1:1s between contributors and mentors, combined with active conversations on PRs, RFC issues, and our project-internal Slack. All these channels helped to keep the communication flowing and ensure that everyone was on the same page. However, it's crucial to try to be responsive. Personally, I could have been better at responding to PRs and questions given how quickly the time flies by, with GSoC being over before you know it!

Post-GSoC Engagement Strategies

After GSoC ends, it's essential to keep contributors engaged in order to build a sustainable community. Continue holding regular office hours, offer additional project ideas, or even invite selected GSoC contributors to mentor the next round of participants. This will go a long way toward creating a sense of belonging and long-term commitment.

Common Pitfalls to Avoid

• Overwhelming Newcomers: Don't assign tasks that are too complex or lacking adequate documentation.
• Inadequate Support: Ensure mentors are available and can provide adequate guidance.
• Poor Documentation: Avoid outdated or incomplete documentation which can create barriers to entry.
• Insufficient Community Interaction: Foster a sense of community and two-way communication.

To provide an illustrative example of where we fell prey to the pitfalls above, a number of contributors working on Windows machines initially struggled with setting up their local development environment. Because the core stdlib team primarily develops on MacOS and Linux, we are largely unaware of the needs and constraints of Windows users, and our contributing guidelines largely reflected that ignorance. Needless to say, telling people to just use Ubuntu shell was not sufficient. We could have saved ourselves a lot of back and forth by (a) providing preconfigured dev containers, (b) investing the time necessary to create more comprehensive documentation, and (c) having a quick onboarding session over a higher bandwidth medium than chat.

Advice for Contributors

• Early Engagement: Interact with the community and start working on beginner-friendly issues early on. If you start contributing before the application period and show your commitment to the project, you will stand out as a proactive candidate during the selection process. This is probably the biggest hack to get selected for GSoC.
• Invest in Project Familiarity Early On: Before contributing code, take time to read through old issues, PR discussions, and any architectural documentation available. Understanding the project's historical context can help avoid misunderstandings and improve the relevance of your contributions.
• Prioritize Code Quality and Documentation: Don't rush to make as many contributions as possible. Take your time to write high-quality code and back it up with sufficient documentation and test cases. Especially in stdlib, we place a high priority on ensuring consistency throughout the codebase, so the more your contributions look and feel like stdlib, the more likely your contributions will be accepted. This attention to detail will set you apart from others who may focus solely on quantity and ignore project conventions.
• Clear Communication: Don't hesitate to ask questions and seek guidance from mentors and the community. Organizations may be overwhelmed with applications, so stepping up and answering questions on the community forums can help you stand out as well.
• Ask for Feedback: Throughout the GSoC program, ask for and incorporate feedback from project mentors. During the GSoC application phase, contributors who clearly demonstrate an ability to receive and act on feedback will stand out. It can be frustrating for project reviewers to repeat the same feedback across multiple PRs, especially concerning project style and conventions. Make it a goal to reduce the number of reviewer comments on each PR. Clean PRs requiring little-to-no review feedback significantly improve the odds of you setting yourself apart from the pack.
• Respect Maintainer Time: Be respectful of maintainer time. GSoC can be highly competitive, and, for many, GSoC acceptance is a meaningful resumé item. Recognize, however, that maintainers often have obligations and jobs outside of their open source work. Sometimes it just isn't possible to immediately review your PR or answer your question, especially toward the end of the GSoC application period. You can significantly improve the likelihood of a response if you heed the advice above; namely, invest in project familiarity early on, prioritize code quality and documentation, and incorporate feedback. Maintainers are human, and they are more likely to invest in you, the more you show you care about them.
• Time Management: Plan your time effectively to meet project milestones and deadlines. The time will fly by, and you don't want to be scrambling to complete your project at the last minute. Break down your project into smaller tasks, and set realistic goals for each week. Where possible, be strategic in your planning, such that, if one task becomes blocked, you can continue making progress by working on other tasks in parallel. If you encounter obstacles, reach out for help sooner rather than later. Being proactive not only ensures you stay on track but also demonstrates your commitment and initiative.
• Participate Beyond Code: Engage in discussions beyond code contributions. Once you have familiarized yourself with the project, gotten up to speed on how to contribute, and successfully made contributions to the codebase, help other newcomers by participating in community channels, answering questions, and directing them to appropriate resources. Not only does this show that you are invested in the community, but it also helps reduce maintainer burden—something which is unlikely to go unnoticed.
• Be Adaptive and Open to Change: Sometimes your initial project plan may not work out as expected. Be flexible and willing to adjust your project scope or approach based on feedback and evolving project priorities.

Acknowledgments

Our heartfelt thanks go out to everyone involved in this year's GSoC, from the mentors and contributors to the broader community, and last but not least, to Google. We're excited to build on the momentum from this summer and look forward to seeing what the future holds for stdlib!

If you're interested in becoming a part of our growing community or exploring the opportunities GSoC can provide, visit our Google Summer of Code repository and join the conversation on our community channels. We're always excited to welcome new contributors!

And if you're just generally interested in contributing or staying updated, be sure to check out the project repository. Don't be shy, and come say hi. We'd love for you to be a part of our community!

Philipp Burckhardt is a data scientist and software engineer securing software supply chains at Socket and a core contributor of stdlib.

stdlib is an open source software project dedicated to providing a comprehensive suite of robust, high-performance libraries to accelerate your project's development and give you peace of mind knowing that you're depending on expertly crafted, high-quality software.

If you've enjoyed this post, give us a star 🌟 on GitHub and consider financially supporting the project. Your contributions help ensure the project's long-term success, and your continued support is greatly appreciated!