Cohere AI Hackathon #3 Summary

During the covid pandemic, a big lesson learned was that students in k-12 education did not do well with online learning. Teachers were given mere weeks to prepare lessons, and they struggled to make engaging content. This project uses the CoHere API and live transcription, done entirely in the browser, to create engaging messages for the students. The AI will "listen" to a presenter's words, find keywords, search wikipedia, find a closely matching topic, rephrase the topic in a "fun" way, and then display this to the user.

Techronic II

an AI-system capable of debating complex topics.

Runtime Error

https://krusnabalar-bridgedoc-frontendsrcmydoc-jwe73t.streamlitapp.com/ INSPIRATION: When was the last time you had an uncomfortable sensation that you didn’t know how to describe? You look it up on google, WebMD and ask onreddit, but find yourself just as uninformed and even more stressed than before. You end up calling a clinic or hospital to speak with a doctor, and find yourself having to wait a week until the next available appointment. The general public is not trained to be aware of and describe the symptoms they might be going through. The way we describe our sensations can vary enormously, often we use idioms and other figures of speech. The stress and struggle of being unable to understand our body’s pain can be frustrating. Even in speaking with doctors, there’s often misunderstandings, and a lot of back and forth, until the doctor can finally understand the patient’s symptoms. In the field of medicine and healthcare, that kind of subjectivity and unpredictability can be dangerous, inefficient, and costly. That got our team to wonder: What if there’s a way to effectively predict a patient’s symptoms based on their own description in a fraction of the time at no cost to the patient? BMJ Journal published a study which performed research on clinical text to extract mental health symptoms and using a classification NLP model, citing the automatability of the symptom detection process as being a credible way to approach this issue. SOLUTION: Introducting BridgeDoc, the tool that doctors and the general public can use to understand and identify their symptoms and diseases. BridgeDoc will use classification tools provided by co:here to detect and identify the specific symptoms, for the knowledge of the doctor, and possible disease diagnoses, for the knowledge of the doctor and the patient. It will allow an ease of communication between a doctor, clinic, or hospital with the patients by using a model trained with colloquial descriptions of symptoms to identify the likelihood of the patient’s symptoms. COMPETITIVE ADVANTAGE: Companies like WebMD, Mayo Clinic, DearDoc, Mercury Healthcare all lack a way to enhance user inquiry and streamline the communication between doctors and patients. BridgeDoc equips users and medical businesses to prevent the struggle and misunderstandings involved in translating a patient’s description of their issue and the doctor’s knowledge of what exactly those symptoms are, helping them efficiently pin point the most likely solutions. This has not been used in a professional medical sense for patients and can help edge over competitors in a significant way with a high quality symptom prediction model. REVENUE AND EXPENSES: There are two B2B (doctors, hospitals) and B2C (online website clients) solution that BridgeDoc provides. B2B solutions will be provided initially per user at a contract price based on the user report and needs of the doctor/hospital. B2C users will have free access. Expenses will be website hosting when moved to a different website that is more customized, as well as to access data that is more reliable by finding methods to get a secure access to it. Research expenses for improving model prediction will also be there. KEY METRICS: We will track the following key metrics Growth: number of doctor contracts acquired per month, number of users accessing website Engagement: use tools like Hotjar to track website interaction (user remains anonymous), track number of searches made per user Marketing: to measure success for our marketing efforts, we will evaluate cost per acquisition, cost per clicks, and understanding the trends in impression to generate better marketing assets iteratively. We will also research marketing tacics of companies like Zocdoc who created an industry tool used by doctors. Product: we will track and receive product feedback dynamically using tools like hotjar to understand user engagement with the product (what parts of the website are being used, how long they spend on the website, etc. HOW WE BUILT IT: We built everything with python. Using jupyter notebooks we tested out co:here’s endpoints, integrated them into the boilerplate provided by LabLab and co:here. The app is deployed using streamlit. Data was collected from various tools such as reddit and google search scrapers, forums. To reach co:here’s requirement for a minimum of 250 examples to train the classify model, due to scarcity of data, we used co:here’s generate tool to build more examples by fine-tuning the model and using specific input phrases that generate reliable results. WHAT'S NEXT FOR BRIDGEDOC We would love to see BridgeDoc to be a standalone tool that can be integrated with online tools for clinics and doctors with private practices, as well as hospital which often deal with issues of patient capacity limits, to automate the report creation by listing possible symptoms and diagnoses automatically. This would require adding a co:here-trained chat-bot that can extract the information from the user in a friendly, secure, and reliable way, similar to how a doctor might on the phone, and produce a report based on the user’s profile (gender, age, previous conditions, etc.) to improve the symptom and disease prediction. Additionally, we leveraged insights from our mentor Ervin, and the “How to get funding from your startup” workshop by Pawel Czech and Mathias Asberg to understand business needs and identify the gaps in what’s currently being offered in our product. It’s important to focus our efforts to specific medical domains to improve accuracy, user retention, and help market and get clients. And the user base can expand by changing the architecture of our model to have a bigger tool that operates like the following: Take input from user and classify what medical domain the query is in. Based on the prediction confidence levels, use the top medical domains and send the user input to specialized classify models trained for the specific domain. Get top predictions from those results of the symptoms as well as the disease. This does not narrow down the user base, and more importantly, it provides improved symptom prediction and more reliable results.

Voyagers

Better tags for short story recommendations using cohere's generate feature! Take in the URL of a story you enjoyed and receive a list of three other stories that you may like to read in return!

Artificially intelligent

Icebreaker questions are often hard to answer, but it is (arguably) even harder to come up with good ones. This app both generates a list of icebreakers given a type (e.g., creative, funny, deep), then judges the difficulty of those questions using a classifier. Another use case of this app could be to better understand potential users of a product. The users' answers could help a company with asking more targeted survey questions and the like. The question generation uses LM prompting on the `xlarge` Cohere pretrained LM. The difficulty classifier is an SVM that inputs embeddings generated through Cohere's API. The code for querying the model APIs and serving the web app is only 135 lines long—showing that anyone could build it, irrespective of coding ability.

Colorless Green Ideas

Solution is the the NozApp -- the App that knows everything about everyone! It analyzes tons of information from Open Sources and Media, so that it can collect and summarize all info about everyone! So when we first open the app, we are asked to enter the username of a twitter user. Once we enter the username, the app checks if the username is valid. This is done so the app can collect information of a valid user based on their Tweets. The app takes sometime to process and collect the data, while sending a request to analyze all the contents. It may take from seconds to minutes, depending on the amount of Tweets the user has. Once it's done anlyzing, the app gets the results requested and shows it as a list of user data, including generated hashtags, the contents summary, the user's recent mood and matching emojis. Just paste identification info about them (nickname or real name) -- and NozApp will collect and summarize all info about the subject: Summary about the person Hashtags, describing this person All URLs of that person Current mood and more other info! Stay tuned! This app will turn the game!

VideoClub

Wohinderland is a powerful NLP application to find vacation destinations, powered by co:here classify. Users simply need to type in their vacation ideas, such as about the weather or about the activities, and the app will give recommendations on places to match the ideas. For example, if the input is “wine, beach, and party”, current prototype will suggest “Bali island” as the place to go. We use classify to try few-shot learning and fine-tuning on collected travel text data. For the prototype, we only limit to seven regions globally and this can be further improved with more data to fine-tune the model. The application has business values in travel industry especially in this post-pandemic era!

BenJanMian Buton

Perfect Prompt is a one stop shop for prompt engineering and image generation. Currently, generating images with prompts for free can be costly with respect to time and money. Perfect Prompt allows you to experiment with your prompt before submitting it several times to a image generation model. Just type in a prompt, and Perfect Prompt will generate an image as well 10 variations of the input prompt with a given art style. First, Perfect Prompt identifies the art style using cohere classify, and matches it to one of five. Then, this style is added to an internal prompt, which asks cohere generate to create variations of the input prompt. These prompts are then ranked again by the classifier, and only the top ten are returned. The user may use any details from these to create their next prompt, and generate another image!

AP

The idea behind this project is to generate recipes from ingredients to avoid wastage of the Food. A great and easier look through to understand the app better just refer to the presentation. In the workflow plan we have two categories to understand. The very first one is "Data". In here the app first scrape about 100 recipes. For each recipe, it gets the list of ingredients as list without quantities. Then it cluster(/embed) ingredients lists and make categories based on embedding clusters. For the second part we have "Usage" where we first take as user input list of ingredients and classify ingredients list according to closest cluster. Then we build prompt from cluster and ingredients list. For the Uses and Scope : (1) It is a resourceful app that can contribute to Sustainable development. (2) Can be a time saver for people who loves to create new recipes especially chefs. (3) Can come out as a treasure for travellers who love to try new cuisines and recipes. (4) Can also become a great online platform for foodies and help them in becoming pro chef who can cook different cuisines.

Memory Makers

The idea is to be able to create an app that lets users add documents they would like to read which are then vectorized to create a search functionality. The app supports inbuilt features like clarifying hard-to-understand phrases in a certain context, generating keywords that would help the user remember better and generating queries that would test the understanding of the reader. The app also creates a network of all the documents that are added to it (using the embeddings).

readandlearn

Recommendation letter in a simpler and quicker way. Сreating a praise letter about another person based on uploaded information from their personal pages and questionary, using common points of intersection between the two people in their studies, work or business.

HDT

Chatty Businessmen is a chat support app, which uses Classify by Cohere to automate appropriate responses to the customers. This app will mainly be used by small businesses, and the main aim of the project is to make the owners of small business workload less.

Super Mario Bros

AI Assisted Intelligent Inbox solution to automatically categorize your emails based on your feedback and preferences, saving you hours of manual triaging, and making you more productive, efficient and effective

Megatron