Botframework-webchat: Feature: Make thumbnail of upload image into channelData
@handsomeyao77 Please see the workaround described here: microsoft/botbuilder-dotnet#2143 (comment)
Original comment:
Hi team,
I get an frequency error, the pictures can't be rendered in chat window, it's broken.
I view this picture src attribute in html:
src="https://bcdirectlineauseast.blob.core.windows.net/at27055/LSfhGXesbMy-original"
The webchat version is V3.
We have multiple chat dialog in the same time, if we switch between these dialog, the image message in chat window often broken.
To Reproduce
Steps to reproduce the behavior:
- Create more than 2 chat dialog
- Chatting with any one(Dialog1), send text or image to him/her
- Then switch to another one(Dialog2), send text or image to him/her
- Now switch back to 2# (Dialog1), the image which your sent is broken in chat window.
Please kindly help me on this issue, thanks very much!
shunyh
All 20 comments
Seems related to https://github.com/microsoft/BotFramework-WebChat/issues/2084
tdurnford
on 2 Jul 2019
Unfortunately v3 of Web Chat is in sunset mode right now and we're not performing maintenance that doesn't fall under security issues for v3.
Could you try out your bot on a basic v4 web page made by following our sample 01.a? This is just to test if your errors are persisting to v4 of Web Chat. If so, we'll be able to investigate the problem and file this as a bug. Otherwise, my immediate fix recommendation would be to use v4 through CDN or MyGet until we get v4 available via embed, which will hopefully be in the upcoming weeks.
tdurnford
on 2 Jul 2019
@tdurnford , thanks your quickly response.
I try to upgrade webchat version to V4, it seems this error still exists:
Besides, the image can't be show as itself size, it looks like limited in 480px*240 px.
shunyh
on 3 Jul 2019
@handsomeyao77 I'm having a difficult time reproducing this issue. Can you share the files you are uploading? It looks like you are sending a text file and a png. Also, double check the file size. Direct Line limits attachments to 4MB.
tdurnford
on 3 Jul 2019
@handsomeyao77 I was able to reproduce the error. The issue only occurs if you pass the Direct Line secret to Direct Line instead of exchanging it for a token first. Try using the code snippet below to test your bot.
<!DOCTYPE html>
<html lang="en-US">
<head>
<title>WebChat</title>
<script src="https://cdn.botframework.com/botframework-webchat/latest/webchat.js"></script>
<style>
html, body { height: 100% }
body {
margin: 0;
}
#webchat {
height: 100%;
}
</style>
</head>
<body>
<div id="webchat" role="main"></div>
<script>
(async function() {
const webChatSecret = '<DIRECT_LINE_SECRET>';
const res = await fetch('https://directline.botframework.com/v3/directline/tokens/generate', {
method: 'POST',
headers: {
'Authorization': `Bearer ${webChatSecret}`,
'Content-Type': 'application/json'
},
body: JSON.stringify({ user: { id: 'dl_user_123' }})
});
const { token } = await res.json();
const directLine = window.WebChat.createDirectLine({ token });
window.WebChat.renderWebChat({
directLine,
}, document.getElementById('webchat'));
})().catch(err => console.log(err));
</script>
</body>
Thank you @stevkan for recommending this fix.
tdurnford
on 4 Jul 2019
@tdurnford , thanks your solution.
I used token instead secret to init DirectLine, just like the sample your provided
Unfortunately, it occurs again, the image src attribute in html is: https://bcdirectlineeasia2.blob.core.windows.net/at27104/8RLaaFI7rYl-original
shunyh
on 4 Jul 2019
@tdurnford , this bug is really urgent for me, could you give me more helps?
shunyh
on 5 Jul 2019
@handsomeyao77 Please see the workaround described here: https://github.com/microsoft/botbuilder-dotnet/issues/2143#issuecomment-508830039
EricDahlvang
on 5 Jul 2019
@EricDahlvang , thank you!
I read microsoft/botbuilder-donet#2143 related info, this is different from our scenario.
Some times we upload image successfully:
But if I refresh our bot page or switch to other same host page , the image gets broken, the request URL is invalid.
We save the images in our storage too, but we can't render images in chat window, all the messages in chat window are from bot-framework service.
We have to provide other page to show the images to user. It's not convenient to us to process huge business cases in short time.
shunyh
on 8 Jul 2019
Hi @handsomeyao77
The best solution to workaround what you are experiencing is to create a thumbnail of the uploaded file and add it to channelData, then set the contentUrl to that channelData in DIRECT_LINE/INCOMING_ACTIVITY as demonstrated in the linked example.
EricDahlvang
on 8 Jul 2019
Hi@EricDahlvang , thanks your response.
I am trying the workaround your mentioned: create a downscale image then add to channelData.
Now, I am using this sample React-web-Chat-Redux to create a listener to listen upload file event.
But I found that this sample has two ReactWebChats component.
One in App.js, the other in webchat.js.
Why do we have to init two WebChats? We only one WebChat in our page.
shunyh
on 10 Jul 2019
There is really just one instance of the ReactWebChat component in the project. The component created in the WebChat.js file acts a container for the ReactWebChat component imported from the BotFramework-WebChat dependency. All of the logic in this file is responsible for creating the DirectLine Connection and the Custom Store Middleware. In theory, all of this code could be moved to the App.js file, but that wouldn't be the best design for larger scale apps.
The App.js file imports the component created in WebChat.js - the one that is acting as a container - and adds it to the App. The ReactWebChat component in this file is different from the ReactWebChat component in the WebChat.js file and should probably be renamed to just WebChat or WebChatContainer to avoid confusion. If you would like, you change the component name in the App.js file in your project to help clarify the difference.
App.js
import { connect } from 'react-redux';
import React from 'react';
import WebChatContainer from './WebChat';
class App extends React.Component {
render() {
const { props: {
backgroundColor,
dispatch
} } = this;
return (
<div id="app" style={{ backgroundColor }}>
<WebChatContainer appDispatch={ dispatch } />
</div>
)
}
}
export default connect(
({ backgroundColor }) => ({ backgroundColor })
)(App)
Hope this helps!
tdurnford
on 10 Jul 2019
Hi @EricDahlvang, here are some summaries what i met share to you.
1. Create a listener.
2. Listener implement .
export default function(dispatch) {
return () => next => action => {
if (action.type === 'DIRECT_LINE/POST_ACTIVITY') {
if (action.payload.activity && action.payload.activity.attachments) {
const downscaledImages = Promise.all(
action.payload.activity.attachments.map(
({ contentType, contentUrl }) => /^image\//.test(contentType) ? downscaleImage(contentUrl,
contentType, 480) : null
)
);
console.log(downscaledImages);
(action.payload.activity.channelData || (action.payload.activity.channelData = {})).thumbnails =
downscaledImages;
}
} else if (action.type === 'DIRECT_LINE/INCOMING_ACTIVITY') {
const activity = action.payload.activity;
if (activity && activity.attachments && activity.channelData && activity.channelData.thumbnails) {
action.payload.activity.attachments = activity.attachments.map((attachment, index) => ({
...attachment,
contentUrl: activity.channelData.thumbnails[index] || attachment.contentUrl
}));
console.log(activity.attachments);
}
}
return next(action);
};
}
3. Upload an image, render in chat window successfully, the contentUrl looks like bot-framework-service, not a base64 data.
4. But if i refresh the page, the contentUrl is invalid, not shown as a base64Data, so image broken.
Is there something wrong?
I think this workaround will convert image to base64 instead use bot-framework Url as img src. Is it right?
shunyh
on 10 Jul 2019
Append some related functions:
async function downscaleImage(imageURL, contentType, maxSize) {
const image = document.createElement('img');
const imageLoadPromise = eventToPromise(image, 'load');
image.src = imageURL;
alert(image.src);
await imageLoadPromise;
const canvas = document.createElement('canvas');
canvas.height = image.height > image.width ? maxSize : ~~(image.height / image.width * maxSize);
canvas.width = image.width > image.height ? maxSize : ~~(image.width / image.height * maxSize);
alert(canvas.height);
alert(canvas.width);
canvas.getContext('2d').drawImage(image, 0, 0, canvas.width, canvas.height);
const dataURL = canvas.toDataURL(contentType);
return dataURL;
}
function eventToPromise(target, name) {
return new Promise((resolve, reject) => {
const handler = event => {
target.removeEventListener(name, handler);
resolve(event);
};
}
target.addEventListener(name, handler);
});
}
shunyh
on 10 Jul 2019
@tdurnford Thanks your explanation, it really helps me a lot.
shunyh
on 10 Jul 2019
@cwhitten We talked about making thumbnail in Web Chat via <canvas>.
Could you give me some suggestions in this topic? E.g. do we need to use Web Worker if available?
compulim
on 10 Jul 2019
We're going to defer the logic into a WebWorker on browsers that support it - if the client is on a browser that does not fully support the isolation to run inside a WebWorker we will run the same logic on the main thread. cc @compulim
cwhitten
on 29 Jul 2019
Hi team,
Thanks for your workaround: https://github.com/microsoft/BotFramework-WebChat/issues/2138#issuecomment-509316793, we fixed the image broken after refresh issue.
But, we encountered the other image related error, image src is a default placeholder:
just like this : https://github.com/microsoft/BotFramework-WebChat/issues/2152.
The new error is different from the old one, should I create a new issue on GitHub?
shunyh
on 30 Jul 2019
Hi Team,
We are facing the same default placeholder issue. Do we have any ETA for the fix? Also, please share any known issues/ limitations once the fix is deployed.
Thanks!
aamalhot
on 2 Aug 2019
The PR #2206 for this issue is pending for review.
It will generate thumbnails and save it as part of the activity (i.e. channelData). The downscaling is verified to work in IE11 and all modern browsers including Safari.
compulim
on 2 Aug 2019
Related issues
naveen-vijay
路
4Comments
GewoonMaarten
路
3Comments
vikramdadwal
路
3Comments
stevengum
路
4Comments
AndreMantas
路
4Comments