Evaluation of AI-Assisted Coding and Computational Research Infrastructure
University of Sheffield
The acceleration of computational research through artificial intelligence has transitioned from an experimental capability to a fundamental requirement for institutional competitiveness. For an institution of the caliber of the University of Sheffield, the deployment of these tools necessitates a rigorous evaluation framework that balances researcher productivity with institutional security.
This report provides a multi-dimensional assessment of thirteen leading AI-assisted coding and research tools, specifically analyzed for their suitability within the University’s ecosystem, including the Stanage and Bessemer HPC clusters and health-related data governance requirements.
1. Comprehensive Profiling of AI Coding Assistants
GitHub Copilot and the Microsoft Ecosystem
GitHub Copilot remains the market-leading AI pair programmer. Its “Agent Mode” is particularly relevant for research, allowing for autonomous task completion across multiple files.
| Criterion | GitHub Copilot Profile Details |
|---|---|
| Primary Function | General-purpose AI pair programming and repository-wide reasoning. |
| Key AI Features | Inline completions, Copilot Chat, Agent Mode, and PR summaries. |
| Languages & Environments | Python, C++, Java, Rust, Go; VS Code, JetBrains, Vim/Neovim. |
| Licensing & Cost | Free for verified students/teachers; Business (39/mo). |
| Data Training Policy | No training on Business/Enterprise data; code is not persisted. |
| HPC Compatibility | High; integrates with Remote-SSH and code-server on Sheffield nodes. |
| Catalogue Status | Approved - Recommended Standard |
Note: Distinct from Microsoft Copilot (Office 365). GitHub Copilot is a technical tool for the software development lifecycle.
Google Gemini Code Assist and Google Colab
Excels in cloud-native development with a massive 128K-token context window.
| Criterion | Google Gemini & Colab Profile Details |
|---|---|
| Primary Function | Gemini: Enterprise coding assistant; Colab: Collaborative notebooks. |
| Data Training Policy | Customer data and prompts are not used to train shared models. |
| SSO Support | Fully supported via Google Workspace for Education and Cloud IAM. |
| Catalogue Status | Approved for GCP-heavy research and collaborative teaching. |
Cursor and the Challenge of AI-Native IDEs
A fork of VS Code that rebuilds the editor’s core for deep, index-level access.
- Status: Restricted
- Risk Profile: Identified vulnerabilities (CVE-2025-54135) and telemetry concerns regarding UK GDPR compliance for sensitive research.
Tabnine and Privacy-First Enterprise Deployment
The premier choice for regulated environments requiring maximum data sovereignty.
| Criterion | Tabnine Profile Details |
|---|---|
| Primary Function | Privacy-centric assistant with VPC/On-premises deployment options. |
| Data Training Policy | Ephemeral processing; code is never stored or used for base models. |
| HPC Compatibility | High; natively supports on-premises deployment on cluster nodes. |
| Catalogue Status | Approved - Recommended for Sensitive Data projects. |
Amazon Q Developer (formerly CodeWhisperer)
Deeply integrated into the AWS ecosystem; includes IP indemnification for Pro tier users.
- Catalogue Status: Approved for AWS-centric research groups.
JetBrains AI Assistant
Deeply integrated for users of PyCharm, CLion, and IntelliJ IDEA.
- Key Feature: “Zero Data Retention” (ZDR) policy.
- Catalogue Status: Approved for researchers using the JetBrains ecosystem.
Continue and Ollama: The Open-Source Stack
The state-of-the-art for private, local AI coding.
| Criterion | Continue + Ollama Profile Details |
|---|---|
| Primary Function | Open-source bridge (Continue) and local model host (Ollama). |
| Data Training Policy | 100% private; no data ever leaves the institutional boundary. |
| HPC Compatibility | High; can be deployed on GPU-enabled worker nodes. |
| Catalogue Status | Recommended Standard for local-first, private development. |
2. Research Infrastructure Integration
HPC Integration (Stanage / Bessemer)
The primary mechanism for enabling these IDEs is the vscoderemote_sheffield_hpc repository, allowing code-server to run on worker nodes.
Standardized Remote Access Workflow:
- Preparation: Load modules (
code-server/4.16.1on Stanage). - Security: Generate SSL certificates via
setup_ssl_ca_server_client.sh. - Connectivity: Spawn Slurm batch jobs via local scripts.
- Resilience: Utilize
reconnect_infofiles for persistent sessions.
Data Governance for Sensitive Research
A decision-making framework based on code/data sensitivity:
| Sensitivity Tier | Recommended Governance Model | Approved Tools |
|---|---|---|
| Tier 1: Clinical | Air-gapped / Local-Only | Tabnine (Self-hosted), Ollama |
| Tier 2: Sensitive | Enterprise Cloud + Zero Retention | Deepnote (w/ BAA), Gemini Enterprise |
| Tier 3: Public | Standard Cloud + Policy Control | GitHub Copilot, Google Colab |
3. Recommendations and Catalogue Status
Priority Recommendation Table
| Tool | Recommended Status | Primary Rationale |
|---|---|---|
| GitHub Copilot | Recommended Standard | Most mature; free academic tier; robust controls. |
| Tabnine Ent. | Recommended (Sensitive) | Only viable solution for air-gapped health research. |
| Ollama + Continue | Recommended (Local) | Zero-cost, high-privacy for individual labs. |
| Deepnote (Edu) | Recommended (Collab) | Superior real-time collaboration; free for researchers. |
| JupyterHub | Recommended Standard | Fundamental for institutional reproducibility. |
| Cursor | Restricted Approval | Permitted for individuals; security risks for scaling. |
Priority Implementation List
- Immediate Standard: Procure GitHub Copilot Enterprise for all centrally-funded RSE and faculty roles.
- Privacy Pathway: Establish a “Sensitive Data Toolkit” centering on Tabnine Enterprise and Ollama for NHS/GDPR Special Category data.
- HPC Enhancement: Mainstream the
vscoderemote_sheffield_hpcscripts through workshops and dedicate high-VRAM GPU nodes (24GB-48GB) for AI inference.