Nouvelle publication

Rechercher

Article
· Mai 15, 2024 2m de lecture
Open Exchange

Retrieve images using vector search (2)

#Embedded Python #SQL #Vector Search #Vue.js #InterSystems IRIS for Health

You need to install the application first. If not installed, please refer to the previous article

Application demonstration

After successfully running the iris image vector search application, some data needs to be stored to support image retrieval as it is not initialized in the library.

Image storage

Firstly, drag and drop the image or click the upload icon, select the image, and click the upload button to upload and vectorize it. This process may be a bit slow.

This process involves using embedded Python to call the CLIP model and vectorize the image into 512 dimensional vector data.

ClassMethod LoadClip(imagePath) As %String [ Language = python ]
{
import torch
import clip
from PIL import Image

device = "cuda" if torch.cuda.is_available() else "cpu"
model, preprocess = clip.load("ViT-B/32", device=device)

image = preprocess(Image.open(imagePath)).unsqueeze(0).to(device)

with torch.no_grad():
    image_features = model.encode_image(image)
return str(image_features[0].tolist())
}

View vector data

Viewing Vector Data in the Management Portal

Vector query

The image vector query process uses VECTOR_COSINE to retrieve the similarity of vector data, which can be achieved through the following SQL.

select top ? id Image,VECTOR_COSINE(TO_VECTOR(ImageVector,double),(select TO_VECTOR(ImageVector,double) from VectorSearch_DB.ImageDB where id= ? )) Similarity from VectorSearch_DB.ImageDB order by Similarity desc

 

After uploading multiple images, select query and the image with the highest similarity to the left will be displayed on the right. Click next to view the next page. The similarity between the two images will gradually decrease with each click.

 

This application is a demo of image vector retrieval. If you are interested, you can check Open Exchange or visit Github to see more information.

3 Comments
Discussion (3)2
Connectez-vous ou inscrivez-vous pour continuer
Article
· Mai 15, 2024 4m de lecture
Open Exchange

Vector search + GenAI + InterSystems technologies with Banksia Global

#Artificial Intelligence (AI) #Angular #Embedded Python #Generative AI (GenAI) #Interopérabilité #Caché #Ensemble #InterSystems IRIS #InterSystems IRIS for Health

 

Hi all! Here I would like to share how we use vector search and GenAI with InterSystems technology. As an example, I'll describe BG-AppealAI project, which our company submitted to the InterSystems Vector Search, GenAI and ML Contest. BG-AppealAI application can write an appeal if you upload an insurance contract and the insurance company’s letter with a refusal to pay medical expenses. Of course, we are aware that at the moment AI has not reached such a level as to create ready-made legal documents. However, it's already possible to create tools to serve as assistants for specialists or ordinary users. Likewise, our application is an AI assistant that answers the question: "Is an appeal possible?" and creates a draft letter of appeal, if possible. 

Explore our application in action by following this link: https://appealai.cloud.banksia.global/#/welcome

We've also created a video showcasing our app: https://www.youtube.com/embed/0p6FvZpzaaA

How BG-AppealAI works

Insurance contracts and rejection letters vectorization: to be able to better assist with appeals, AI needs a context of the denial. One of the possible ways we can provide a context is by citing relevant parts of insurance contracts. Upon receiving an insurance contract, BG-AppealAI breaks it down into parts, converting each part into a multi-dimensional vector. These vectors are stored in an InterSystems IRIS database, allowing fast and accurate retrieval when needed. 

Vector Search:  when you upload a rejection letter to the app, BG-AppealAI converts it to the vector, this is called the vectorization process.  BG-AppealAI then uses IRIS vector search functionality to compare vectors extracted from the rejection letter with vectors extracted from the relevant insurance contract.

AI-Powered Appeal Generation: having identified potential context for appeal, BG-AppealAI sends a request to the generative AI service to assess the possibility of drafting an appeal. If it is deemed appropriate, AppealAI sends a new request in order to create a draft letter of appeal.

 

What's under the hood of the BG-AppealAI

Traditionally, our application consists of backend and frontend parts, where the frontend is developed on Angular framework and the backend on InterSystems IRIS. The video about our application has already shown the work of the frontend part of the application and a brief overview of the product, here I want to focus on the implementation of Interoperability Production. This is where we can clearly see the application of vector search and generative AI. 

So, there are the following elements in our system:
1) ConversionToVector - process that converts document text into vectors using LangChain, and invokes process GetText and operations SaveVector, SaveDocument for the corresponding actions.

 

2) GetText - process that parses data from a file into a stream, for further text processing and analysis. Here there is opportunity to work with 2 types of extensions: docx and pdf, as the most common ones for documents. 

 

3) VectorSearch - process that performs a vector search and looks for matches between documents. Also, it is in this business process that methods are called to write promts to the AI and analyze rejection letters. 

 

Note, that here we used a language model developed by OpenAI and the LangChain framework. All business processes are implemented using embedded python as it provides easy access to utilization to use LangChain and write promts to OpenAI.
4) AppealIn - service that receives the rejection file and sends it for analysis.
5) DocumentIn - service that receives the insurance contract file and sends it for vectorization.
6) OpenAiOut - operation that makes a request to a generative AI - OpenAI, receives promt, returns a response from GenAI.
7) SaveDocument - operation that stores the document file in the database.
8) SaveVector - operation that stores vectors in the database.
9) SaveAppeal - operation that gets the text of the appeal and stores it in the database.

The interaction of all the above elements is well shown by the visual trace. Visual trace of taking an insurance contract, dividing it into vectors, and storing the vectors in a database:

 

Visual trace of receiving a rejection letter, sending requests to OpenAI, and saving the appeal letter if an appeal is possible:

 

This is an example of a rejection letter:

 

This is an example of an appeal written by BG-AppealAI:

 

We hope that our open source project will be useful to this community and help you implement innovative technologies in your projects. Our team would like to thank InterSystems for the opportunity to work with cutting-edge technologies. We're eager to engage with your thoughts and questions in the comments below. Let's continue pushing the boundaries of what's possible with technology together! Stay tuned for more updates and insights from our team.

If you enjoyed getting to know BG-AppealAI and its features and would like to support us, please take a moment to vote for our app here https://openexchange.intersystems.com/contest/36. Thank you!

3 Comments
Discussion (3)5
Connectez-vous ou inscrivez-vous pour continuer
Question
· Mai 15, 2024

How to insert data to Third party Database？

#Ensemble

hi !

now I want to insert data to third party database directly,  Link Table wizard how to insert  except by sql? 

s obj=##class(YBFXJC.LABINFO).%New()
  ;s obj.ID="1"
  s obj.HISID="1"
  s obj.APPYNO="1"
  s obj.HOSPITALID="1"
  s obj.BILGDRNAME="1"
  s sc=obj.%Save()
  w sc_$System.Status.GetErrorText(sc),!

 there is error  <CANNOT SET THIS PROPERTY>zSaveData+3^YBFXJC.TestSaveObj.1

4 Comments
Discussion (4)3
Connectez-vous ou inscrivez-vous pour continuer
Discussion
· Mai 15, 2024

【Coding Puzzle】 "Esthetic Numbers"

#Code Snippet #ObjectScript #Python #Autre

Hello,

I would like you to propose this challenge.

It has been created by the CodeWars community here: https://www.codewars.com/kata/6523a71df7666800170a1954/python

I will copy and paste the description:

 

🔡🟢 DESCRIPTION:

A number is Esthetic if, in any base from base2 up to base10, the absolute difference between every pair of its adjacent digits is constantly equal to 1.

num = 441 (base10)
// Adjacent pairs of digits:
// |4, 4|, |4, 1|
// The absolute difference is not constant
// 441 is not Esthetic in base10

441 in base4 = 12321
// Adjacent pairs of digits:
// |1, 2|, |2, 3|, |3, 2|, |2, 1|
// The absolute difference is constant and is equal to 1
// 441 is Esthetic in base4

Given a positive integer num, implement a function that returns an array containing the bases (as integers from 2 up to 10) in which num results to be Esthetic, or an empty array [] if no base makes num Esthetic.

Examples

10 ➞ [2, 3, 8, 10]
// 10 in base2 = 1010
// 10 in base3 = 101
// 10 in base8 = 12
// 10 in base10 = 10

23 ➞ [3, 5, 7, 10]
// 23 in base3 = 212
// 23 in base5 = 43
// 23 in base7 = 32
// 23 in base10 = 23

666 ➞ [8]
// 666 in base8 = 1232

 

The initial Python code is just:

def esthetic(num):
    pass

 

And the Unit Tests are:

import codewars_test as test
from solution import esthetic

@test.describe("Fixed tests")
def fixed_tests():
    @test.it('Basic test Cases')
    def basic_test_cases():
        test.assert_equals(esthetic(10), [2, 3, 8, 10], "Example #1")
        test.assert_equals(esthetic(23), [3, 5, 7, 10], "Example #2")
        test.assert_equals(esthetic(666), [8], "Example #3")
        test.assert_equals(esthetic(13), [5, 6])
        test.assert_equals(esthetic(1), [2, 3, 4, 5, 6, 7, 8, 9, 10])
        test.assert_equals(esthetic(9), [4, 7, 9, 10])
        test.assert_equals(esthetic(74), [])
        test.assert_equals(esthetic(740), [4, 6, 9])
        test.assert_equals(esthetic(928), [])
        test.assert_equals(esthetic(259259), [9])
        test.assert_equals(esthetic(883271), [])
        test.assert_equals(esthetic(1080898), [7])
        test.assert_equals(esthetic(1080899), [])

 

 

 

 

My solution was:

def esthetic(num):
    def to_base(n, base):
        """Convert number n to a given base and return the digits as a list."""
        digits = []
        while n:
            digits.append(n % base)
            n //= base
        return digits[::-1]  # reverse the list to get the correct order

    def is_esthetic_in_base(digits):
        """Check if the given list of digits is esthetic."""
        for i in range(1, len(digits)):
            if abs(digits[i] - digits[i - 1]) != 1:
                return False
        return True

    esthetic_bases = []

    for base in range(2, 11):
        digits = to_base(num, base)
        if is_esthetic_in_base(digits):
            esthetic_bases.append(base)

    return esthetic_bases

 

 

How would you like to solve it using ObjectScript or any other prefered programming language?

4 Comments
Discussion (4)1
Connectez-vous ou inscrivez-vous pour continuer
Article
· Mai 15, 2024 2m de lecture
Open Exchange

Retrieve images using vector search (1)

#Embedded Python #Vector Search #Vue.js #InterSystems IRIS

Hi Community,

In this article, I will introduce my application iris-image-vector-search.
The image vector retrieval demo uses IRIS Embedded Python and OpenAI CLIP model to convert images into 512 dimensional vector data. Through the new feature of Vector Search, VECTOR-COSINE is used to calculate similarity and display high similarity images.

Application direction of image retrieval  

Image retrieval has important application scenarios in the medical field, and using image retrieval can greatly improve work efficiency. Image retrieval can also be applied in the following fields, such as:

 

  • Image retrieval systems can be used to search for medical image data related to their research topic, for data analysis, pattern recognition, and research, accelerating the process of scientific research.
  • The images in the medical imaging database can be used for the education and training of medical students. Through image retrieval, students can search and compare different types of cases, deepening their understanding of disease characteristics and diagnostic methods.
  •  Image retrieval can be used to assist doctors in diagnosis. By comparing medical imaging data of patients (such as X-rays, CT scans, MRI, etc.) and providing reference images of similar cases through a knowledge base, doctors can quickly obtain relevant information and improve diagnostic accuracy.  

How to use it

Prerequisites

Make sure you have git and Docker desktop installed.

Installation

  • Clone/git pull the repo into any local directory

git clone https://github.com/yueshan239/iris-image-vector-search.git

  • Open the terminal in this directory and run

docker-compose build

    This process will take some time

 

  • Run the IRIS container

docker-compose up -d

 

  • Open the terminal in `vue` directory and run

docker-compose build

  • Run the nginx container

docker-compose up -d

 Visit the address below

http://localhost:8080/

Accessing this page indicates that we have successfully run it.

Discussion (0)1
Connectez-vous ou inscrivez-vous pour continuer