Scully: Page is getting generated without data

[kunjee17]

🐞 Bug report

I have read access on services but page is getting generated without data

Description

Below is angular code. Getting called in ngInit

this.subSink.add(
      this.faqService.getAll().subscribe(
        (f) => {
          this.panels = f.map((x) => ({ active: false, ...x }));
        },
        (error) => {
          console.error(error);
        }
      )
    );

faq service is fetching data from firestore. That have universal read access.

🔬 Minimal Reproduction

💻Your Environment

Angular Version:

11.0.9

Scully Version:

^1.0.0

🔥 Exception or Error

[SanderElias]

Can you provide a reproduction repo for this?
My guess is that the faqService is returning a hot observable. When Scully is running this might take too long, so Scully is done extracting the HTML the results are not there?
Without access to a way to reproduce this issue, there is no way for me to know the actual cause.

[kunjee17]

@SanderElias is it ok if I add you to gitlab private project? I am guessing your guess is right. Because I did face similar issue while doing form validation.

Or I can get you some sample project in day or two, on github. whatever you like. Is there any trace mechanism that I can turn it on and if you want logs for same?

[SanderElias]

@kunjee17 I would prefer the GitHub repo, as that would give someone having the same problem an insight into what is going on.

[kunjee17]

Sure I ll do it. Basically I was skeptical about firebase access. I ll create temp project for same. Give me a day and it will be ready.

[yringler]

@kunjee17, I think I'm seeing an issue where if the stream is passed through a pipe, data isn't being saved.
Are you using pipes?

[kunjee17]

@yringler yes. I'm using pipe. But subscribing at very end.

[yringler]

TL;DR: I think we're having the same issue, see links for code.

Interesting.
I found in my case that scully only saved the data if I passed in the observable directly, but not if I first made a pipe.
Even though I also subscribed after.
Here's the commit which fixed it for me https://github.com/Health-Freedom/site/commit/e50a6eaef5dc6a86580c3df704a6643735283c03

A simpler diff when I ran into the problem again (I thought a simpler pipe would work) is here: https://github.com/Health-Freedom/site/commit/97ed0da53c79234f6cc1dc9e3c397fc23811b81f

Come to think of it, I'm probably dealing with a hot observable - from what I saw (see this commit for my workaround https://github.com/Health-Freedom/site/commit/a6002739ae90c5dc76935af4adfdfaea78602b22), Apollo Angular (which I'm using) runs the network queries even before the returned observable is subscribed to, which is I suppose the definition of a hot observable.

[kunjee17]

@yringler I'll give it a try and get back in a day. This should work.

[yringler]

Great! As I learn more about jamstack, I'm probably going to move to embracing it more fully

1. Only scully builds makes API calls
2. relying on ScullyRoutesService for any data I need
3. saving that data with transfer state
4. rebuilding with webhooks when data changes

which will avoid the whole issue, and also means I can start locking down my graphql server etc quite a bit more.

But depending on the domain, that probably isn't a solution for everyone.

[kunjee17]

Sorry I couldn't able to make it work. Here is reproduction repo . Run the application and go to faq page.

[SanderElias]

@kunjee17 I'l work on this after I finish my current task

[kunjee17]

No hurries @SanderElias . Code is working only, data is getting fetched runtime instead of scully time that is the only issue. So, it is not really a blocker as of now.

[SanderElias]

@kunjee17 The problem is that your firebase driver runs outside of ngZone.
Because of that, Scully does extract the HTML before the data is available.
For use-cases like this, we have a manualIdle control. That way, you can tell Scully when the page is ready.
I did a PR to your reproduction that show how that would work.
first I updated your component to this:

@Component({
  selector: 'app-faq',
  templateUrl: './faq.component.html',
  styleUrls: ['./faq.component.less'],
})
export class FaqComponent {
  loading = true;
  panels$ = this.tss
    .useScullyTransferState(
      'faq',
      this.faqService.getAll().pipe(
        take(1), // as it is hot, we need to take only the first one
        /**
         * as firebase runs outside of zone, we need to manually tell Scully the page is "ready"
         * the setTimeout makes sure Angular has some time to paint the page, before we
         * will "scrape"
         */
        tap(() => setTimeout(() =>this.ims.fireManualMyAppReadyEvent(),10)), //
      )
    )
    .pipe(
      tap(() => (this.loading = false)),
      /** if you want to subscribe to future updates you can merge the 'hot'
       * service back in, or use a subject to push to. */
      // mergeMap(()=> this.faqService.getAll()),
      map((faq) => faq.map((x) => ({ active: false, ...x } as Panel)))
    );
  constructor(
    private faqService: FaqService,
    private tss: TransferStateService,
    private ims: IdleMonitorService
  ) {}
}

And in your config file I updated the route to use ManualIdele checking. like this:

  routes: {
    '/faq': {
      type: 'default',
      manualIdleCheck: true
   },
  },

After those changes, your app works like expected.
I'm going to close this issue because there is no action Scully can take. I will reopen the issue if there is a reason to do so!
Feel free to keep discussing in here. If you have more issues, don't hesitate to open up a new issue.

[kunjee17]

Thanks @SanderElias that was quick. I ll surely have a look and keep you posted.