Fashion is an ever-evolving industry that requires constant adaptation and innovation to stay relevant. But staying up-to-date with the latest trends can be a daunting task.

In this blog post, I’ll explore how ResNet-50 can be leveraged to build robust fashion recommendation systems that can provide personalized fashion choices to customers, making their shopping experience seamless and enjoyable.

It’s a captivating realm, and one of the latest technological advancements in the industry is the use of deep learning algorithms for fashion recommendation systems.

In this blog, we’ll explore how to use the ResNet-50 model for building a fashion recommendation system.

For this point in time, we created one Streamlit webpage on a local system to see the 10 recommended fashion product images which look similar to the query image.

What is ResNet-50?

ResNet-50 is a deep convolutional neural network that was introduced by Microsoft Research in 2015. It is a powerful model that has been trained on millions of images and can recognize a wide range of objects, including fashion items. This makes it an ideal choice for building a fashion recommendation system. For a detailed explanation, check out my other blog on ResNet-50.

We won’t fully train on the ResNet-50 model from scratch(as it’s already trained on the ImageNet dataset). We use a transfer learning concept here, and just simply extract the features by passing dataset images (credits: Param Aggarwal). The dataset consists of 44k images of fashion products.

dataset images — Credits: Param Aggarwal

We’ll pass 44k images to the ResNet-50 model and take the output of the last conv. block before avg. pooling. The resultant output would be a 2D array of size 44k*2048 where each image has a 2048 dimensional vector called features.

Why Streamlit?

Streamlit provides a simple and intuitive API to create web applications using only a few lines of Python code.

Streamlit is an open-source Python library that enables developers to quickly and easily create web applications for data science and machine learning projects. With Streamlit, developers can build interactive web-based applications that allow users to visualize, explore, and analyze data in real-time.

Streamlit includes built-in support for a wide range of data visualization and analysis tools, including charts, tables, maps, and machine learning models.

For further details, check out their official website.

Plan of Action

Import model: Import model ResNet-50 using keras with preprocessing_input to just process the image.

from tensorflow.keras.applications.resnet50 import ResNet50, preprocess_input
############################ Defining Model##############################################
model=ResNet50(weights='imagenet',include_top=False, input_shape=(224,224,3))
model.trainable=False
model=tf.keras.Sequential([model,GlobalMaxPool2D()])
model.summary()

Extract features: Pass 44k images to ResNet-50 model, extract features, and then flatten the features to convert them into 1D vectors. Extracted features and images have to be stored in pickle files so that during recommendation, we pass the index of resultant features to get images in return.

############### One time Code: need to extract features of 44k images, U can run this ######

def image_preprocess(path,model):

img=image.load_img(path, target_size=(224,224))

img_arr=image.img_to_array(img)

ex_img_arr=np.expand_dims(img_arr,axis=0)

pre_pr_img=preprocess_input(ex_img_arr)

result=model.predict(pre_pr_img).flatten()

normal_result=result/norm(result)

return normal_result

path=r'C:\Users\TanishSharm\Desktop\Fashion_recom_sys\myntradataset\images'

images=[os.path.join(path,files) for files in os.listdir(path)]

pickle.dump(images,open('images.pkl','wb'))

feature_list=[]

for file in tqdm(images):

feature_list.append(image_preprocess(file, model))

pickle.dump(feature_list,open('fetaures.pkl','wb'))

Load extracted features: Get recommendations for given query points using distance metrics.

################################Loading Stored Features and images##################################

file_img=pickle.load(open(r'C:\Users\TanishSharma\Desktop\Fashion_recom_sys\images.pkl','rb'))

feature_list=(pickle.load(open(r'C:\Users\TanishSharma\Desktop\Fashion_recom_sys\fetaures.pkl','rb')))

Save Images: We use Streamlit to make this system. Create one upload button that will provide a browse option, then click on upload image which will be saved into the uploads folder in your local system.

st.title('Fashion Recommender system')## title of the webpage

###################### Method to Save Uploaded Image into local############################

def Save_img(upload_img):

try:

with open(os.path.join('uploads',upload_img.name),'wb') as f:

f.write(upload_img.getbuffer())

return 1

except:

return 0

########### To display upload button onto screen######################

upload_img=st.file_uploader("Choose an image")

Generate recommendation: Now features get stored and will pass new query images to the ResNet-50 model. It’ll generate features that will flatten further. Here, we use Euclidean distance metrics to find out the recommended 10 Nearest Neighbors or look for similar images. Below are two methods to extract feature and prod recommendations:

######################## Method to Extract features of new query image#######################

def feature_extraction(path,model):

img=image.load_img(path, target_size=(224,224))# Load image in size of 224,224,3

img_arr=image.img_to_array(img)# storing into array

ex_img_arr=np.expand_dims(img_arr,axis=0)## Expanding the dimension of image

pre_pr_img=preprocess_input(ex_img_arr)## preprocessing the image

result=model.predict(pre_pr_img).flatten()### to make 1d vector

normal_result=result/norm(result)## Normalize the result using norm func from linalg(numpy)

return normal_result

def prod_recom(features, feature_list):

neb=NearestNeighbors(n_neighbors=10,algorithm='brute',metric='euclidean') #using brute force algo here as data is not too big

neb.fit(feature_list)## fit with feature list

dist, ind=neb.kneighbors([features])# return distance and index but we use index to find out nearest images from stored features vector return ind

We’ll call these functions to extract features, get 10 recommendations, and create sections of recommended images in the Streamlit app. Find below the code for the same:

### Condition to check if image got uploaded then call save_img method to save and preprocess image followed by extract features and recommendation

if upload_img is not None:

if Save_img(upload_img):

st.image(Image.open(upload_img))

st.header("file uploaded successfully")

features=feature_extraction(os.path.join("uploads",upload_img.name),model)

progress_text = "Hold on! Result will shown below."

my_bar = st.progress(0, text=progress_text)

for percent_complete in range(100):

time.sleep(0.02)

my_bar.progress(percent_complete + 1, text=progress_text) ## to add progress bar untill feature got extracted

ind=prod_recom(features, feature_list)# calling recom. func to get 10 recommendation

### to create 10 section of images into the screen

col1,col2,col3,col4,col5,col6,col7,col8,col9,col10=st.columns(10)

##for each section image shown by below code

with col1:

st.image(Image.open(file_img[ind[0][0]]))

with col2:

st.image(Image.open(file_img[ind[0][1]]))

with col3:

st.image(Image.open(file_img[ind[0][2]]))

with col4:

st.image(Image.open(file_img[ind[0][3]]))

with col5:

st.image(Image.open(file_img[ind[0][4]]))

with col6:

st.image(Image.open(file_img[ind[0][5]]))

with col7:

st.image(Image.open(file_img[ind[0][6]]))

with col8:

st.image(Image.open(file_img[ind[0][7]]))

with col9:

st.image(Image.open(file_img[ind[0][8]]))

with col10:

st.image(Image.open(file_img[ind[0][9]]))

# st.text("Using Spotify ANNoy")

# df = pd.DataFrame({'img_id':file_img, 'img_repr': feature_list})

# f=len(df['img_repr'][0])

# ai=AnnoyIndex(f,'angular')

# for i in tqdm(range(len(feature_list))):

# v=feature_list[i]

# ai.add_item(i,v)

# ai.build(10) # no of binary tress want to build more number of tree more accuracy

# neigh=(ai.get_nns_by_item(0,5))

# with col1:

# st.image(Image.open(file_img[neigh[0]]))

# with col2:

# st.image(Image.open(file_img[neigh[1]]))

# with col3:

# st.image(Image.open(file_img[neigh[2]]))

# with col4:

# st.image(Image.open(file_img[neigh[3]]))

# for i in range(len(neigh)):

# with st.columns(i):

# st.image(Image.open(file_img[neigh[i]]))

else:

st.header("Some error occured")

Find the full code of .py file in my GitHub profile.

Steps

Create a new project in your IDE (PyCharm, Spyder, Vscode, etc.)
Paste your downloaded dataset from the given resource inside the same folder
Execute the given two steps in the above plan of action section and store features in local machine inside the same folder.
Open cmd-> activate python env-> Streamlit run main.py

Browse any fashion product image and wait for a couple of seconds. The 10 look-alike products are shown below:

Recommendations can be improved based on distance metrics. You can try different metrics like Manhattan distance, Minkowski distance, or Hamming Distance, and see the results. Or you can try annoy index (used by Spotify in music recommendation) to find out the best-recommended products.

Conclusion

A fashion recommendation system using ResNet-50 can provide a personalized shopping experience for users and help fashion retailers increase sales. By analyzing the features of fashion items, the system can recommend similar items that are more likely to be of interest to the user.

As deep learning algorithms continue to advance, we can expect to see more sophisticated fashion recommendation systems in the future.

Fashion product recommendation system using ResNet-50

Tanish Sharma