Skip to main content

Homework 4: ChatBot Builder Prototype

In this homework, you will implement a chatbot builder prototype with a local LLM in Vue.js. In your chatbot builder a user will be able to select different characteristics of system prompts to change the behavior of the chatbot for a specific task.

info

Your laptop must have at least 10 GB free in order to complete this assignment. (If not, the LLM will fail to load.)

You will need a laptop that supports WebGPU to complete this assignment.

The course material needed to complete this assignment was covered in classes 11, 12, 13, and 14.

Learning Objectives​

  • Experiment with different system prompt characteristics and evaluate the responses generated by a local LLM.
  • Reduce environmental impact by using a local LLM instead of a cloud-based LLM.
  • Configure a development environment's AI tooling with AGENTS.md.
  • Develop AGENTS.md to improve the accuracy of AI-generated code and reduce token usage.
  • Through prompt engineering and effective task selection, demonstrate that a tiny local LLM can perform reasonably well on a specific task.
  • Implement a prototype of a task-specific chatbot using an AI coding agent.
  • Employ app design patterns.
  • Deploy a prototype as an installable Progressive Web App (PWA).

Help & Support Resources​

We want you to get the help and support you need. Use the course's support resources for help with this assignment.

Git Repository & Codespace​

  1. Create your assignment repository.

    Visit https://landrace.infosci.cornell.edu/courses/info4340-2026fa/repos/hw4 in your browser to create your assignment repository.

  2. Open your assignment repository in GitHub.

    After creating your repository, visit https://landrace.infosci.cornell.edu/courses/info4340-2026fa/repos/hw4 in your browser and follow the link to open the repository on github.com

  3. Open your assignment repository as a codespace.

Submission Requirements​

Professionalism

Professionalism means meeting the client's requirements. Not doing less and not doing more.

Your submission should meet the requirements specified for this assignment and no more (or less). If you do more than the requirements, you will receive a 0 for the assignment. If you do less than the requirements, you will receive a 0 for the assignment.

Your assignment should meet the following requirements for credit:

  • Submit your own original work (design and code) for this assignment.

    No credit is provided for submitting design and/or code that is taken from the course-provided examples.

  • Your code should be original. However, the structure of the code will mirror the class examples.

    The structure of your code will align with the class examples. This is to be expected.

  • You are required to use the methods and techniques covered in class.

    Important

    No credit is provided for using methods that are not covered in this class.

    You may not use any methods that have not been covered in class prior to the release of this assignment.

  • Your assignment's prototype should be visible in the browser and functional upon launching the development web server using Run Without Debugging.

    If your assignment does not launch using this method or is not visible in the browser after launching the development server, we are unable to provide any credit.

  • All files should be in the location specified in this document for credit.

    Professionalism is important. Incorrectly placed files will not be graded; no partial credit is provided either.

  • Follow the submission instructions below.

    Submit all materials to your GitHub repository's main branch for this assignment. When you're finished stage, commit, and push your submission to GitHub. Then fill out the submission form to complete your submission.

    Failure to complete the submission form will result in 0 credit; no submission form = no submission. No leniency. No exceptions.

Sprint 1: Local LLM ChatBot​

Credit: 0 points.

Goal: Implement a local LLM chatbot prototype in Vue.js.

Requirements​

  • Use WebLLM and the provided class module to implement a local LLM chatbot prototype in Vue.js.
  • Your git commit history should have at least one commit for this sprint. (The sprint commits must be in order for credit.)

Instructions​

  1. Install the WebLLM package in your assignment repository.

    npm install @mlc-ai/web-llm

  2. Import the class provided module in App.vue to use a local LLM in your Vue.js app.

    import { llmEngine, status, progress, Status } from "./services/localLlm";

  3. In App's sendPrompt() function, send the messages to the local LLM using the llmEngine.generate() method.

    try {
      await llmEngine.generate(messages.value, {
        onResponse: (text) => (response.value = text),
      });
    } catch (error) {
      console.error("Failed to generate a response:", error);
      hasResponseError.value = true;
    }

  4. In App's stopPrompt() function, stop the local LLM using the llmEngine.stop() method.

    llmEngine.stop();

Submission​

You will not submit this sprint for credit.

Sprint 2: Prompt Engineering Experimentation​

Credit: ~10 points. Completion credit provided for meeting the requirements below. No partial credit is provided.

Goal: Experiment with different system prompt characteristics and evaluate the responses generated by the local LLM.

Requirements​

You will need to modify the provided chatbot prototype to experiment with different system prompt characteristics.

  • Experiment with each of the following system prompt characteristics by modifying the systemPrompt:

    • Persona: Defines the model's identity and role (e.g., helpful assistant, expert programmer).
    • Safety & Ethics Focus: Includes directives to avoid harmful, biased, or inappropriate content.
    • Behavioral Guidance: Instructs on tone (e.g., polite, neutral), helpfulness, and interaction style.
    • Capability Awareness: Outlines general abilities while acknowledging limitations (e.g., knowledge cutoff).
    • Response Formatting: Include instructions on how to format responses (e.g., concise, detailed, with examples).

  • Document your experiments and findings in the README.md file in your assignment repository. Include the following information for each experiment:

    • The system prompt you used.
    • A sample user prompt you sent to the local LLM.
    • The response generated by the local LLM.
    • Your evaluation of the response (e.g., was it interesting, accurate, helpful, etc.?).

  • Your experiments should be substantive; no credit provided for unserious or trivial experimentation.

  • Your git commit history should have at least one commit for this sprint. (The sprint commits must be in order for credit.)

  • Do not modify the Vue.js interface in this sprint.

Instructions​

warning

If you encounter an error in your console that looks something like this:

Failed to execute 'add' on 'Cache': Cache.add() encountered a network error

You've run out of disk space. Free up at least 10 GB of disk space and then clear your browser cache. (Or clear the cache for your development server's URL.)

  1. Modify the systemPrompt in App.vue to experiment with different system prompt characteristics. Do not modify the Vue.js interface in this sprint.

    For example, you might experiment with different personas:

    const systemPrompt = ref(`Your name is Alice, a superhero. You're witty and sarcastic.`);
    const systemPrompt = ref(`You are Bob. You're perceived as super confident, because you're loud and brash.`);

    Get a sense of how the system prompt characteristics affect the responses generated by the local LLM.

  2. Take note of which of your experiments are successful and provide interesting responses. Document your experiments and findings in the README.md file in your assignment repository. Include the following information for each experiment:

    • The system prompt you used.
    • A sample user prompt you sent to the local LLM.
    • The response generated by the local LLM.
    • Your evaluation of the response (e.g., was it interesting, accurate, helpful, etc.?).

Submission​

Have this sprint completed as you'll need it for the next class. However, you will not submit this sprint yet. You will submit the sprint as part of the last sprint's submission.

Sprint 3: Agent Configuration​

Credit: ~10 points. Completion credit provided for meeting the requirements below. No partial credit is provided.

Goal: Configuring your development environment to provide greater accuracy (and hopefully use fewer tokens) when utilizing AI tools within Codespaces.

Requirements​

For this sprint, develop an AGENTS.md file that meets the following requirements:

  • An AGENTS.md file should exist at the root of your repository. (Additional AGENTS.md files in subdirectories will not be graded.)
  • The AGENTS.md file should provide clear, but concise instructions for the LLM to follow when generating code.
  • The AGENTS.md file should be well-structured and employ best-practices.
    • Places commands at the top of the file. (e.g. check that the app builds with npm run build)
    • Provides code examples over explanations.
    • Provides clear boundaries (e.g., "What files can the AI touch and not touch?")
    • Provides specifics about your tech stack (e.g., "What version of Vue.js?").
  • The AGENTS.md file should responsibly consume tokens by providing only the necessary information for the LLM to generate code.
  • Your agent configuration should produce code that is consistent with the existing code provided in the repository.
  • Your agent configuration should model the methods we have used to author components in Vue.js this semester.
    • The agent should impose the same constraints on the code it generates as we have imposed on our own code. (However, the code may use computed() as it's conceptually similar to watch().)
    • For example, we have a preference for emitted events over using the default v-model event. The agent should follow this preference when generating code.
    • As another example, we also have a preference for minimal custom CSS styling.
    • Not all our conventions and norms will be provided here. Part of your task is to identify these conventions and norms and provide them in your AGENTS.md file.
  • Your git commit history should show revision to your AGENTS.md file.

Instructions​

  1. Create an AGENTS.md file at the root of your repository.

  2. Draft your AGENTS.md file to provide clear, but concise instructions for the local LLM to follow when generating code.

    Use the best practices outlined in class and in the AGENTS.md example provided in class.

  3. Commit your draft AGENTS.md file to your repository.

  4. Test your AGENTS.md file by using it to generate code for your chatbot builder prototype.

    Your chatbot builder will need to let the user select a persona for how the chatbot responds. Use GitHub Copilot to generate the code to select a persona in the app. (Your app will need to let the user select between at least 3 different personas.)

    Evaluate the response generated by GitHub Copilot and determine if it meets your expectations. If not, revise your AGENTS.md file and try again.

    For example, if you're seeking to add a select dropdown to your app, did the agent generate/use a reusable select dropdown component? Then did it integrate that select component into the app for persona selection? Is that component styled similarly to the existing app? Does the component emit an event when the user selects a persona? Does the app listen for that event and update the system prompt accordingly?

  5. Once you've refined your AGENTS.md file, use GitHub Copilot to generate the code for letting the user select three output formats.

  6. Commit your AGENTS.md file and the code generated by GitHub Copilot to your repository.

    Sync your repository with GitHub to ensure your commits are on the server.

Submission​

Have this sprint completed as you'll need it for the next class. However, you will not submit this sprint yet. You will submit the sprint as part of the last sprint's submission.

Sprint 4: Task Specific ChatBot Experimentation​

Credit: ~10 points. Completion credit provided for meeting the requirements below. No partial credit is provided.

Goal: Identify a task that a tiny local LLM can perform well given its limitations.

Requirements​

For this sprint, identify a task that a tiny local LLM performs reasonably well provided a well engineered system prompt for a task-specific chatbot.

  • Define a clear and specific task for your chatbot.
  • Draft a well engineered system prompt that provides reasonable performance for your task-specific chatbot.
  • At the top of README.md, document your task-specific chatbot.
    • Provide a brief description of the task your chatbot performs.
    • Provide a system prompt that provides reasonable performance for your task-specific chatbot.
    • Provide a sample user prompt that demonstrates the task your chatbot performs.
    • Provide the response generated by the local LLM for your sample user prompt.
    • Provide the average response time for your task-specific chatbot.
  • No credit is provided for general purpose chatbots, like ChatGPT; your chatbot must be task-specific for credit.
  • Your git commit history should have at least one commit for this sprint. (The sprint commits must be in order for credit.)

Instructions​

An LLM only "knows" the information it was initially trained on. If an LLM was not trained on a specific topic, it likely will perform poorly when asked to generate tokens about that topic. However, a well engineered system prompt can provide the LLM with the context it needs to perform reasonably well on a task.

The tiny local LLM provided in this assignment was trained in 2023, and even still it doesn't "know" all that much. But it has been fine-tuned to follow instructions, and it does so reasonably well.

Your task is to learn how to better control an LLM through effective prompt engineering, rather than relying on an expensive frontier language model that requires little expertise to operate.

However, the tiny local LLM isn't all that fast. The longer your system prompt, the slower the chatbot becomes. Engineer a system prompt that provides the LLM with the context it needs to perform reasonably well on a task, but is also concise enough to provide reasonable response times.

  1. Using the task-specific brainstorm chatbot activity from class, experiment with different tasks that the tiny local LLM can perform reasonably well given a well engineered system prompt for a task-specific chatbot.

    The objective of this experimentation is to practice prompt engineering and to control an LLM's behavior through your own critical thinking.

  2. Through experimentation, select a task that seems the most promising and refine your system prompt to improve the quality of the responses and the response time.

  3. Document your task-specific chatbot at the top of README.md in your assignment repository. Include the following information:

    • A brief description of the task your chatbot performs.
    • A system prompt that provides reasonable performance for your task-specific chatbot.
    • A sample user prompt that demonstrates the task your chatbot performs.
    • The response generated by the local LLM for your sample user prompt.
    • The average response time for your task-specific chatbot.

  4. Commit your README.md file to your repository and sync to GitHub.

Submission​

Have this sprint completed as you'll need to demo it for a peer in the next class. However, you will not submit this sprint for credit yet. You will submit the sprint as part of the last sprint's submission.

Sprint 5: Prototype a Task Specific ChatBot​

Credit: ~35 points. Completion credit provided for meeting the requirements below. No partial credit is provided.

Goal: Implement a prototype app of your task specific chatbot employing an AI coding agent.

Requirements​

Use the task-specific chatbot and system prompt you developed in the previous sprint, to implement a fully functional prototype of the chatbot by employing an AI Coding agent. Your prototype should meet the following requirements:

  • Develop the prototype using an AI agent like GitHub Copilot.
    • The prototype should be implemented using only the methods and techniques covered in this course.
    • Evaluate the code generated by GitHub Copilot and verify that it meets the expectations of this class. If it fails to do so, revise AGENTS.md as necessary or improve your prompts.
    • Export all AI "chats" and "agent sessions" to an ai-sessions folder at the root of your repository as JSON. (You need not use GitHub Copilot, but if you use another agent, its session history must be importable to GitHub Copilot for credit.)
    • You may not use a web-based/app-based Chatbot tool, like ChatGPT to generate the code for this assignment.
  • Your system prompt(s) should be well engineered and employ the best practices covered in class.
    • No credit is provided for large lists of instructions.
    • No credit is provided for excessive context.
  • All prompt engineering characteristics should be present in your system prompt(s) (persona, safety & ethics focus, behavioral guidance, capability awareness, and response formatting).
  • The prototype should support the user selecting at least 3 different personas and 3 output formats for how the task-specific chatbot responds.
  • Your system prompt should use at least one of the following methods: one-shot, few-shot, or chain-of-thought prompting.
  • Your prototype should dynamically modify the system prompt to only include the necessary information for the LLM to perform the task well, and nothing more.
    • No credit is provided for static system prompts.
    • Long system prompts will result in no credit.
    • No credit is provided for multiple "hard-coded" system prompts that are not dynamically modified based on the user's selections.
  • Your chatbot should "remember" its conversation history and use that history to inform its responses.
  • Your chatbot should provide "new chat" functionality to clear the current chat and start anew.
  • Your git commit history should have at least one commit for this sprint. (The sprint commits must be in order for credit.)

Instructions​

  1. Take a moment and plan what tasks you need to complete to implement a fully functional prototype of your task-specific chatbot. Consider the following:

    • What components will you need to implement?
    • How will you structure your system prompt(s) to provide the LLM with the necessary context for your task-specific chatbot?
    • How will you dynamically modify the system prompt(s) based on the user's selections?
    • How will you implement the "remember" functionality for your chatbot?
    • How will you implement the "new chat" functionality for your chatbot?

  2. Implement a fully functional prototype of your task-specific chatbot using an AI coding agent like GitHub Copilot.

    tip

    Your GitHub Education benefits provide a small AI usage/token budget. In this assignment, you will need to use your AI token budget wisely. It must last you the entire month.

    Careful planning and smart prompting will help you conserve your AI token budget. However, if you exceed your budget, you may need to purchase additional AI usage from GitHub.

    Important

    For credit, you must export all AI "chats" and "agent sessions" to an ai-sessions folder at the root of your repository as JSON.

    If you are using GitHub Copilot, you can do this from the Command Palette then select Chat: Export Chat... to export the current chat/agent session.

  3. Once you have a fully functional prototype of your task-specific chatbot, test it to ensure it meets the requirements outlined above.

    Make sure you test your task-specific chatbot with different user prompts to ensure it performs well on a multitude of possible user prompts.

  4. Commit your code to your repository and sync to GitHub.

Submission​

You will not submit this sprint for credit yet. You will submit the sprint as part of the last sprint's submission.

Sprint 6: Installable ChatBot Builder App​

Credit: ~15 points. Completion credit provided for meeting the requirements below. No partial credit is provided.

Goal: An installable chatbot PWA that employs app design patterns.

Requirements​

For credit, your chatbot builder should be installable as a PWA and employ app design patterns.

  • Your chatbot app prototype should employ app design patterns; no credit is provided for non-app design patterns, including web design patterns.
  • Create an installable progressive web app (PWA).
    • Your PWA should be installable from GitHub Pages (no credit for installable PWAs from the development server).
    • Your PWA should be functional and usable after installing it via Chrome on a laptop device.
  • Your git commit history should have at least one commit for this sprint. (The sprint commits must be in order for credit.)

Instructions​

  1. Refine your app's design to employ only app design patterns.

    • Apps fill the entire window and are responsive to window size changes. (No fixed dimensions, no overflow scrollbars for the app's window, etc.)
    • Apps do not have a "mobile-sized" box centered in the middle of the screen where the app sits surrounded by a sea of whitespace.
    • Apps are not websites!
    • Apps do not use links as buttons. (In apps, clicking a link opens that link in a new browser window.)
    • Apps do not have a big website style heading that says "CHATBOT APP".
    • Apps put the user's task front and center, not the app's branding or marketing.

  2. Make your app installable as a PWA.

  3. Test that your app is installable and functional after installing it via Chrome on a laptop device.

Submission​

Stage, commit and push all changed files in your Git repository to the GitHub server. (All commits should reside on the main branch.)

Complete the submission form for hw4 to submit the assignment.

note

The submission form asks you to check your submission. Checking your work will ensure you receive credit for this assignment. We ask you to check your submission because this is where some students lose points. It's easy to forget something and checking your work prevents the heartache of getting a 0 because your submission wasn't submitted in a way that we can access and grade it.

Contributors​

Copyright © 2026:

  • Kyle Harms
Last updated on